| Author |
Message |
janderuiter
Joined: 24 Jun 2009 Posts: 5
|
Posted: Wed Jun 24, 2009 8:40 pm Post subject: scraping php sites? |
|
|
i am trying to scrape a php site.
i went through the settings but am getting mixed results.
the site is pretty simple.
specific page: http://65.23.133.224/web-v2.php
any ideas?
thanks,
jan |
|
| Back to top |
|
 |
stanbusk Site Admin
Joined: 28 Dec 2005 Posts: 4439
|
Posted: Thu Jun 25, 2009 8:15 am Post subject: |
|
|
| That can be because the site is accessing a database, is this possible? What do you mean with mixed results? |
|
| Back to top |
|
 |
janderuiter
Joined: 24 Jun 2009 Posts: 5
|
Posted: Thu Jun 25, 2009 1:58 pm Post subject: |
|
|
it is accessing a db but in the end, the pages are just html pages. basically a news-like structure where i have a jump page with links to detail pages. the pages themselves require no variables or anything dynamic.
i was hoping the dumper tool would scrape these pages as html, including following the links, and scraping them as well. |
|
| Back to top |
|
 |
stanbusk Site Admin
Joined: 28 Dec 2005 Posts: 4439
|
Posted: Thu Jun 25, 2009 3:02 pm Post subject: |
|
|
| Yes, it is exactly what it is intended to as long as there is no code URLs embedded inside Javascript. Most site are download properly. The problem here is perhaps the fact everything is database centered, I mean the relinking can cause problems. I will try the site. |
|
| Back to top |
|
 |
stanbusk Site Admin
Joined: 28 Dec 2005 Posts: 4439
|
Posted: Thu Jun 25, 2009 3:06 pm Post subject: |
|
|
| I have tried right after opening the preferences and pressing the 'Factory Default' button. With those default settings I am able to download the site and view it offline with my browser. |
|
| Back to top |
|
 |
janderuiter
Joined: 24 Jun 2009 Posts: 5
|
Posted: Thu Jun 25, 2009 3:33 pm Post subject: |
|
|
weird. i started with that (so i thought) before posting here.
Anyway, thanks so much. i will buy your tool next and try for myself.
thanks again! |
|
| Back to top |
|
 |
|