Find and create a list of all the urls of a particular website
You might need to do this if you’re moving to a new permalink structure and need to 301 redirect the pages. For large sites, a lot of time can be saved by making good use of free sitemap generators online and excel.
A bunch of the online sitemap generator tools either ask for your email or have a maximum number of pages that they index. This one: www.xsitemap.com is my favorite since it works well, doesn’t ask for an email, and as far as I can tell, has no limit on the number of pages. I just ran it for a site with 2500 pages. It took about a half hour.
After it’s finished, you have two options for grabbing the data. Either you can copy and paste from the readout or download the file provided.
Once you’ve pasted the data into excel, use the find/replace function to eliminate all the data you don’t need. For 301 redirects, you can get rid of anything extraneous, which means everything prior to the suffix, including the root domain.
301 redirect structure looks like this:
redirect 301 /old/old.htm http://www.you.com/new.htm
