How to scrap an website when you don't know the URL for that

Discussions and Tech Support related to website data extraction, screen scraping and data mining using iMacros.

Moderators: Community Moderators, iMacros Moderators

Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the Google search box (at the top of each forum page) to see if a similar problem or question has already been addressed. This will search the entire contents of the forums as well as the iMacros Wiki.
3. We can respond much faster to your posts if you include the following information:

CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST

Answering your own posts (e.g. attempting to "bump" your topic) drops your topic from the list of unanswered threads, so it may actually receive less views.

How to scrap an website when you don't know the URL for that

by arvendu on Tue Jun 13, 2017 12:20 am

I have this website that I want retrieve information from. Unfortunately I can't show you because it is password protected.

The site is a list of tables that are dynamically created.
they design the link to activate a script. The first link is:
javascript:__doPostBack('ctl00$ContentPlaceHolder1$GridView1$ctl02$LinkButton1','')

The second link is the same, but with the last variable changed ($ctl03')"). It goes from 0 to 30


so is there any possiblity to change url automatic each loop
arvendu
 
Posts: 1
Joined: Tue Jun 13, 2017 12:12 am

Re: How to scrap an website when you don't know the URL for

by chivracq on Tue Jun 13, 2017 9:22 am

arvendu wrote:I have this website that I want retrieve information from. Unfortunately I can't show you because it is password protected.

The site is a list of tables that are dynamically created.
they design the link to activate a script. The first link is:
javascript:__doPostBack('ctl00$ContentPlaceHolder1$GridView1$ctl02$LinkButton1','')

The second link is the same, but with the last variable changed ($ctl03')"). It goes from 0 to 30


so is there any possiblity to change url automatic each loop

CIM...! :mrgreen: (Read my Sig...)

But, yep, compute the "0" (or "00"...?) - "30" part of your URL using 'EVAL()' based on '!LOOP'... :idea:
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all 3 Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
chivracq
 
Posts: 5996
Joined: Sat Apr 13, 2013 6:07 am
Location: Amsterdam (NL)


Return to Data Extraction and Web Screen Scraping

Who is online

Users browsing this forum: No registered users and 2 guests

-->