How to click multiple links on search results page & sav

Support for iMacros. The iMacros software is the unique solution for automating every activity inside a web browser, for data extraction and web testing.
Forum rules
iMacros EOL - Attention!

The renewal maintenance has officially ended for Progress iMacros effective November 20, 2023 and all versions of iMacros are now considered EOL (End-of-Life). The iMacros products will no longer be supported by Progress (aside from customer license issues), and these forums will also no longer be moderated from the Progress side.

Thank you again for your business and support.

Sincerely,
The Progress Team

Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information: CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
Post Reply
Suresh Varma

How to click multiple links on search results page & sav

Post by Suresh Varma » Fri Oct 21, 2005 2:53 am

Is there way to click multiple links on a search results page and save the content of each of the search result page ?

SCENARIO: I have a search form and when a keyword search is performed it would display all the pages that match the search results. I want to open each of the pages that match the search and save the content as a Text File.

Scenario Example and Macro Code: I go to http://www.Dice.com, the advanced job search page, search for a Keyword like "Java" and click the search button. I would get a results page with various jobs that match the criteria, I want to click each of these jobs and save the content and click the "Back to search results" link taking me back to the saerch results page. I want to continue saving every resulting job page like I did the 1st one.

I am using VB script to do this, my first macro would be the searching Macro and I want to second macro to be a click link, save it and take me back to the search results. The second macro would loop for all the links in the search results page.

Here is my first macro.

VERSION BUILD=4310722
TAB T=1
TAB CLOSEALLOTHERS
URL GOTO=http://www.dice.com/
TAG POS=1 TYPE=A ATTR=TXT:Advanced<SP>Search
TAG POS=1 TYPE=INPUT:TEXT FORM=ACTION:/jobsearch/servlet/JobSearch ATTR=NAME:FREE_TEXT CONTENT=java
TAG POS=1 TYPE=SELECT FORM=ACTION:/jobsearch/servlet/JobSearch ATTR=NAME:STAT_PROV CONTENT=1
TAG POS=1 TYPE=SELECT FORM=ACTION:/jobsearch/servlet/JobSearch ATTR=NAME:NUM_PER_PAGE CONTENT=0
TAG POS=1 TYPE=SELECT FORM=ACTION:/jobsearch/servlet/JobSearch ATTR=NAME:DAYSBACK CONTENT=6
TAG POS=1 TYPE=INPUT:IMAGE FORM=ACTION:/jobsearch/servlet/JobSearch ATTR=NAME:HOTRESULTS&&VALUE:

How do I write the second Macro ?

Please help...............


Thanks for reading the long post.
User avatar
Tech Support
Posts: 4948
Joined: Tue Sep 20, 2005 7:25 pm
Contact:

Post by Tech Support » Fri Oct 21, 2005 9:04 pm

The solution will be very similar to the solution described at http://www.iopus.com/iim/help/faq_extract_pages.htm or the sample code found at http://forum.imacros.net/viewtopic.php?t=6 .

The main difference is that the above examples only extract the search result title, whereas you click on the title and then do the extract.
SureshVarma
Posts: 1
Joined: Thu Oct 20, 2005 4:59 pm

How do I click ?

Post by SureshVarma » Fri Oct 21, 2005 9:24 pm

I was able to get till there and was not able to figure out how to click it.

It may be something really simple that I am missing.

This is what I have there

'TAG POS=1 TYPE=A ATTR=TXT:Blue<SP>Advantage<SP>Option<SP>II

TAG POS=1 TYPE=A ATTR=TXT:{{ThisPlanName}}

The first line works and when I comment it and pass the same value from the VBA code, it doesn't. How do I figure this out ?
User avatar
Tech Support
Posts: 4948
Joined: Tue Sep 20, 2005 7:25 pm
Contact:

Post by Tech Support » Fri Oct 21, 2005 10:49 pm

This is the TAG command you can use to select a result

Code: Select all

TAG POS={{myloop}} TYPE=A ATTR=HREF:http://seeker.dice.com/jobsearch/servlet/JobSearch*
The variable {{myloop}} would be 1...x, until all results are extracted

Important note: I created this TAG command by changing the clickmode from automatic to "Use URL". The reason for this is that the TAG commands that are created by default use the TEXT of the link like in

Code: Select all

TAG POS=1 TYPE=A ATTR=TXT:J2EE<SP>Technical<SP>Lead   
TAG POS=1 TYPE=A ATTR=TXT:IBM<SP>OnDemand<SP>Consultant 
TAG POS=1 TYPE=A ATTR=TXT:Java<SP>/<SP>Websphere<SP>developer
This does not help us as the text changes all the time with each new link. However, the basic URL structure stays constant for each posting:

Code: Select all

TAG POS=1 TYPE=A ATTR=HREF:http://seeker.dice.com/jobsearch/servlet/JobSearch?op=101&dockey=xml/f/c/fcd98180e5637d160d090012e206bcaa@activejobs0&c=1&source=1   
TAG POS=1 TYPE=A ATTR=HREF:http://seeker.dice.com/jobsearch/servlet/JobSearch?op=101&dockey=xml/f/a/facca104128bff2cc2fc184fea838491@activejobs0&c=1&source=1   
Now all that needs to be done is to replace the changing part of the URL (session ID) with * and we get

Code: Select all

TAG POS={{myloop}} TYPE=A ATTR=HREF:http://seeker.dice.com/jobsearch/servlet/JobSearch?op=101*
For additional information see also http://www.iopus.com/iim/help/faq_session_id.htm and http://www.iopus.com/iim/help/extract_techtips.htm
toq
Posts: 2
Joined: Tue Jan 24, 2006 5:08 pm

Post by toq » Tue Jan 24, 2006 5:16 pm

wahh nice thing...
simicar
Posts: 2
Joined: Wed Jul 05, 2006 7:16 am

Script developing..

Post by simicar » Wed Jul 05, 2006 7:34 am

I'm writing due to extending the above script and use it as an example.
Here it is in general after updates:

VERSION BUILD=4310722
TAB T=1
TAB CLOSEALLOTHERS
URL GOTO=http://www.dice.com/
TAG POS=1 TYPE=A ATTR=TXT:Advanced<SP>Search
TAG POS=1 TYPE=INPUT:TEXT FORM=ACTION:/jobsearch/servlet/JobSearch ATTR=NAME:FREE_TEXT CONTENT=java
TAG POS=1 TYPE=SELECT FORM=ACTION:/jobsearch/servlet/JobSearch ATTR=NAME:STAT_PROV CONTENT=1
TAG POS=1 TYPE=SELECT FORM=ACTION:/jobsearch/servlet/JobSearch ATTR=NAME:NUM_PER_PAGE CONTENT=0
TAG POS=1 TYPE=SELECT FORM=ACTION:/jobsearch/servlet/JobSearch ATTR=NAME:DAYSBACK CONTENT=6
TAG POS=1 TYPE=INPUT:IMAGE FORM=ACTION:/jobsearch/servlet/JobSearch ATTR=NAME:HOTRESULTS&&VALUE:
TAG POS={{!LOOP}} TYPE=A ATTR=HREF:http://seeker.dice.com/jobsearch/servle ... rch?op=101*


J've got a quesion:
Is there a possibility to LOOP scripting only on the selected fragment, not whole scipt ex.
I need to visit a result page of a search, then save with CPL method each one by entering link privided
- step back to previous page, and use second link & save it (there goes LOOPing all the results)
- and what is the syntax used for "stepping back", sth like javascript:history.back();

Thanks
User avatar
Tech Support
Posts: 4948
Joined: Tue Sep 20, 2005 7:25 pm
Contact:

Post by Tech Support » Tue Jul 11, 2006 10:15 am

Hi,

If you find that you want to loop over parts of macros you should consider how you can split the macro.

Although I don't fully understand your example, it would seem that the first macro should obtain the results page and then a second macro would do the actions for each link looping through each of the links.

In terms of stepping back, most websites provide navigation tools that iMacros can automate, such as a "return to search results" button. Often search results pages don't respond well to clicking the back button on your browser having followed one of the links.

How is this navigation done manually? Do you have to re-do the search each time? If so, then perhaps you do need to loop through the whole macro each time.

Daniel Kerr
iOpus Technical Support
simicar
Posts: 2
Joined: Wed Jul 05, 2006 7:16 am

Post by simicar » Tue Jul 11, 2006 12:51 pm

Hi again.

The splitting macro solution perfectly fits here while redoing the search isn't needed.
It's beacuse the script should obtain each link provided, execute it and save the contents or just maybe put those links in the table as a variable.
Then the script will simply extract each row for a link and save it - but does iMacros support that operation?

Whitch solution will be the best here, that's what I'm asking for and if you could just type the ex. source code I would be grateful.

Thanks
User avatar
Tech Support
Posts: 4948
Joined: Tue Sep 20, 2005 7:25 pm
Contact:

Post by Tech Support » Wed Jul 12, 2006 4:24 pm

Hi,

I have had a look at your website and iMacros should be able to handle it all just fine. In terms of the clicking the links, please see my colleague's code earlier in this thread.

There is a button for navigating back to the results page so I would recommend using that.

It terms of the extraction, it depends exactly how you want it. But if we assume the field headings are constant, you could extract the specific field information by looping through the table

Code: Select all

SET !LOOP 3
EXTRACT POS={{!LOOP}} TYPE=TXT ATTR=<TD<SP>vAlign=top<SP>align=left>*
And the long textual description with a relative extraction

Code: Select all

TAG POS=1 TYPE=B ATTR=TXT:Job<SP>description:    
EXTRACT POS=R1 TYPE=TXT ATTR=<TD<SP>colSpan=2>*
Daniel Kerr
iOpus Support
Post Reply