How to click multiple links on search results page & sav

Support for iMacros. The iMacros software is the unique solution for automating every activity inside a web browser, for data extraction and web testing.

Moderators: Community Moderators, iMacros Moderators

Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the Google search box (at the top of each forum page) to see if a similar problem or question has already been addressed. This will search the entire contents of the forums as well as the iMacros Wiki.
3. We can respond much faster to your posts if you include the following information:

CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST

Answering your own posts (e.g. attempting to "bump" your topic) drops your topic from the list of unanswered threads, so it may actually receive less views.

How to click multiple links on search results page & sav

by Suresh Varma on Thu Oct 20, 2005 7:53 pm

Is there way to click multiple links on a search results page and save the content of each of the search result page ?

SCENARIO: I have a search form and when a keyword search is performed it would display all the pages that match the search results. I want to open each of the pages that match the search and save the content as a Text File.

Scenario Example and Macro Code: I go to http://www.Dice.com, the advanced job search page, search for a Keyword like "Java" and click the search button. I would get a results page with various jobs that match the criteria, I want to click each of these jobs and save the content and click the "Back to search results" link taking me back to the saerch results page. I want to continue saving every resulting job page like I did the 1st one.

I am using VB script to do this, my first macro would be the searching Macro and I want to second macro to be a click link, save it and take me back to the search results. The second macro would loop for all the links in the search results page.

Here is my first macro.

VERSION BUILD=4310722
TAB T=1
TAB CLOSEALLOTHERS
URL GOTO=http://www.dice.com/
TAG POS=1 TYPE=A ATTR=TXT:Advanced<SP>Search
TAG POS=1 TYPE=INPUT:TEXT FORM=ACTION:/jobsearch/servlet/JobSearch ATTR=NAME:FREE_TEXT CONTENT=java
TAG POS=1 TYPE=SELECT FORM=ACTION:/jobsearch/servlet/JobSearch ATTR=NAME:STAT_PROV CONTENT=1
TAG POS=1 TYPE=SELECT FORM=ACTION:/jobsearch/servlet/JobSearch ATTR=NAME:NUM_PER_PAGE CONTENT=0
TAG POS=1 TYPE=SELECT FORM=ACTION:/jobsearch/servlet/JobSearch ATTR=NAME:DAYSBACK CONTENT=6
TAG POS=1 TYPE=INPUT:IMAGE FORM=ACTION:/jobsearch/servlet/JobSearch ATTR=NAME:HOTRESULTS&&VALUE:

How do I write the second Macro ?

Please help...............


Thanks for reading the long post.
Suresh Varma
 

by Tech Support on Fri Oct 21, 2005 2:04 pm

The solution will be very similar to the solution described at http://www.iopus.com/iim/help/faq_extract_pages.htm or the sample code found at http://forum.iopus.com/viewtopic.php?t=6 .

The main difference is that the above examples only extract the search result title, whereas you click on the title and then do the extract.
User avatar
Tech Support
 
Posts: 5003
Joined: Tue Sep 20, 2005 12:25 pm

How do I click ?

by SureshVarma on Fri Oct 21, 2005 2:24 pm

I was able to get till there and was not able to figure out how to click it.

It may be something really simple that I am missing.

This is what I have there

'TAG POS=1 TYPE=A ATTR=TXT:Blue<SP>Advantage<SP>Option<SP>II

TAG POS=1 TYPE=A ATTR=TXT:{{ThisPlanName}}

The first line works and when I comment it and pass the same value from the VBA code, it doesn't. How do I figure this out ?
SureshVarma
 
Posts: 1
Joined: Thu Oct 20, 2005 9:59 am

by Tech Support on Fri Oct 21, 2005 3:49 pm

This is the TAG command you can use to select a result
Code: Select all
TAG POS={{myloop}} TYPE=A ATTR=HREF:http://seeker.dice.com/jobsearch/servlet/JobSearch*

The variable {{myloop}} would be 1...x, until all results are extracted

Important note: I created this TAG command by changing the clickmode from automatic to "Use URL". The reason for this is that the TAG commands that are created by default use the TEXT of the link like in
Code: Select all
TAG POS=1 TYPE=A ATTR=TXT:J2EE<SP>Technical<SP>Lead   
TAG POS=1 TYPE=A ATTR=TXT:IBM<SP>OnDemand<SP>Consultant
TAG POS=1 TYPE=A ATTR=TXT:Java<SP>/<SP>Websphere<SP>developer

This does not help us as the text changes all the time with each new link. However, the basic URL structure stays constant for each posting:
Code: Select all
TAG POS=1 TYPE=A ATTR=HREF:http://seeker.dice.com/jobsearch/servlet/JobSearch?op=101&dockey=xml/f/c/fcd98180e5637d160d090012e206bcaa@activejobs0&c=1&source=1   
TAG POS=1 TYPE=A ATTR=HREF:http://seeker.dice.com/jobsearch/servlet/JobSearch?op=101&dockey=xml/f/a/facca104128bff2cc2fc184fea838491@activejobs0&c=1&source=1   


Now all that needs to be done is to replace the changing part of the URL (session ID) with * and we get

Code: Select all
TAG POS={{myloop}} TYPE=A ATTR=HREF:http://seeker.dice.com/jobsearch/servlet/JobSearch?op=101*


For additional information see also http://www.iopus.com/iim/help/faq_session_id.htm and http://www.iopus.com/iim/help/extract_techtips.htm
User avatar
Tech Support
 
Posts: 5003
Joined: Tue Sep 20, 2005 12:25 pm

by toq on Tue Jan 24, 2006 10:16 am

wahh nice thing...
toq
 
Posts: 2
Joined: Tue Jan 24, 2006 10:08 am

Script developing..

by simicar on Wed Jul 05, 2006 12:34 am

I'm writing due to extending the above script and use it as an example.
Here it is in general after updates:

VERSION BUILD=4310722
TAB T=1
TAB CLOSEALLOTHERS
URL GOTO=http://www.dice.com/
TAG POS=1 TYPE=A ATTR=TXT:Advanced<SP>Search
TAG POS=1 TYPE=INPUT:TEXT FORM=ACTION:/jobsearch/servlet/JobSearch ATTR=NAME:FREE_TEXT CONTENT=java
TAG POS=1 TYPE=SELECT FORM=ACTION:/jobsearch/servlet/JobSearch ATTR=NAME:STAT_PROV CONTENT=1
TAG POS=1 TYPE=SELECT FORM=ACTION:/jobsearch/servlet/JobSearch ATTR=NAME:NUM_PER_PAGE CONTENT=0
TAG POS=1 TYPE=SELECT FORM=ACTION:/jobsearch/servlet/JobSearch ATTR=NAME:DAYSBACK CONTENT=6
TAG POS=1 TYPE=INPUT:IMAGE FORM=ACTION:/jobsearch/servlet/JobSearch ATTR=NAME:HOTRESULTS&&VALUE:
TAG POS={{!LOOP}} TYPE=A ATTR=HREF:http://seeker.dice.com/jobsearch/servlet/JobSearch?op=101*


J've got a quesion:
Is there a possibility to LOOP scripting only on the selected fragment, not whole scipt ex.
I need to visit a result page of a search, then save with CPL method each one by entering link privided
- step back to previous page, and use second link & save it (there goes LOOPing all the results)
- and what is the syntax used for "stepping back", sth like javascript:history.back();

Thanks
simicar
 
Posts: 2
Joined: Wed Jul 05, 2006 12:16 am

by Tech Support on Tue Jul 11, 2006 3:15 am

Hi,

If you find that you want to loop over parts of macros you should consider how you can split the macro.

Although I don't fully understand your example, it would seem that the first macro should obtain the results page and then a second macro would do the actions for each link looping through each of the links.

In terms of stepping back, most websites provide navigation tools that iMacros can automate, such as a "return to search results" button. Often search results pages don't respond well to clicking the back button on your browser having followed one of the links.

How is this navigation done manually? Do you have to re-do the search each time? If so, then perhaps you do need to loop through the whole macro each time.

Daniel Kerr
iOpus Technical Support
User avatar
Tech Support
 
Posts: 5003
Joined: Tue Sep 20, 2005 12:25 pm

by simicar on Tue Jul 11, 2006 5:51 am

Hi again.

The splitting macro solution perfectly fits here while redoing the search isn't needed.
It's beacuse the script should obtain each link provided, execute it and save the contents or just maybe put those links in the table as a variable.
Then the script will simply extract each row for a link and save it - but does iMacros support that operation?

Whitch solution will be the best here, that's what I'm asking for and if you could just type the ex. source code I would be grateful.

Thanks
simicar
 
Posts: 2
Joined: Wed Jul 05, 2006 12:16 am

by Tech Support on Wed Jul 12, 2006 9:24 am

Hi,

I have had a look at your website and iMacros should be able to handle it all just fine. In terms of the clicking the links, please see my colleague's code earlier in this thread.

There is a button for navigating back to the results page so I would recommend using that.

It terms of the extraction, it depends exactly how you want it. But if we assume the field headings are constant, you could extract the specific field information by looping through the table

Code: Select all
SET !LOOP 3
EXTRACT POS={{!LOOP}} TYPE=TXT ATTR=<TD<SP>vAlign=top<SP>align=left>*


And the long textual description with a relative extraction

Code: Select all
TAG POS=1 TYPE=B ATTR=TXT:Job<SP>description:   
EXTRACT POS=R1 TYPE=TXT ATTR=<TD<SP>colSpan=2>*


Daniel Kerr
iOpus Support
User avatar
Tech Support
 
Posts: 5003
Joined: Tue Sep 20, 2005 12:25 pm


Return to General Support & Discussions

Who is online

Users browsing this forum: Google [Bot] and 11 guests

-->