Extract and save a link from webpage

Discussions and Tech Support related to website data extraction, screen scraping and data mining using iMacros.
Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information:CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
Post Reply
pstefanov
Posts: 2
Joined: Tue May 19, 2015 4:14 am

Extract and save a link from webpage

Post by pstefanov » Tue May 19, 2015 4:31 am

Hey all,
first I read many topics from all the net and I can't solve my problem.

Code: Select all

VERSION BUILD=8920312 RECORDER=FX
URL GOTO=webpage.test 
SET !EXTRACT_TEST_POPUP NO
SET !ERRORIGNORE YES
SET !TIMEOUT_STEP 2
TAG POS=1 TYPE=A ATTR=ID:turker EXTRACT=HREF
SAVEAS TYPE=EXTRACT FOLDER=E:\folder\ FILE=Cherveni.txt
The case is:
When open the webpage, the webpage generate link which I want to save in text file. That is done. But sometimes the webpage don't generate a link and in the text file is writen ""#EANF#"". I want to skip this step when there is no link to save. I try some methods with javascript, but I cannot solve it. Please guys, help me :)


Windows 8/Firefox 38.0.1/
Shoop
Posts: 18
Joined: Fri Apr 17, 2015 10:58 am

Re: Extract and save a link from webpage

Post by Shoop » Tue May 19, 2015 11:11 am

javascript and if statements.
or use eval
save extracted to some variable, like var1 or wutever.
then use eval and check if the content of var1 is eanf. if it is repalce eanf with nothing. else just do nothing (no else part)
then just save the var1 to extract and use the same saveas functions
chivracq
Posts: 8523
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Extract and save a link from webpage

Post by chivracq » Tue May 19, 2015 12:42 pm

pstefanov wrote:Hey all,
first I read many topics from all the net and I can't solve my problem.

Code: Select all

VERSION BUILD=8920312 RECORDER=FX
URL GOTO=webpage.test 
SET !EXTRACT_TEST_POPUP NO
SET !ERRORIGNORE YES
SET !TIMEOUT_STEP 2
TAG POS=1 TYPE=A ATTR=ID:turker EXTRACT=HREF
SAVEAS TYPE=EXTRACT FOLDER=E:\folder\ FILE=Cherveni.txt
The case is:
When open the webpage, the webpage generate link which I want to save in text file. That is done. But sometimes the webpage don't generate a link and in the text file is writen ""#EANF#"". I want to skip this step when there is no link to save. I try some methods with javascript, but I cannot solve it. Please guys, help me :)

Windows 8/Firefox 38.0.1/
Shoop wrote:javascript and if statements.
or use eval
save extracted to some variable, like var1 or wutever.
then use eval and check if the content of var1 is eanf. if it is repalce eanf with nothing. else just do nothing (no else part)
then just save the var1 to extract and use the same saveas functions
Yep, a .js Script would be the normal way to do it, but a quick and dirty Solution would be to use 'EVAL()' indeed, but I would rather use it on the File Name than on the Data because the 'SAVEAS' will still add an empty Line otherwise:

Code: Select all

VERSION BUILD=8920312 RECORDER=FX
URL GOTO=webpage.test 
SET !EXTRACT_TEST_POPUP NO
SET !ERRORIGNORE YES
SET !TIMEOUT_STEP 2
TAG POS=1 TYPE=A ATTR=ID:turker EXTRACT=HREF
'SAVEAS TYPE=EXTRACT FOLDER=E:\folder\ FILE=Cherveni.txt
'>
SET SAVEAS_File EVAL("var s='{{!EXTRACT}}'; var x; if(s!='#EANF#'){x=\"Cherveni.txt\";} else {x=\"Safe_to_delete.txt\";}; x")
SAVEAS TYPE=EXTRACT FOLDER=E:\folder\ FILE={{SAVEAS_File}}
(Not tested but I use a similar Technique in one of my Scripts...)

And from time to time, you delete this 'Safe_to_delete.txt' File containing all the '#EANF#''s, or you can include a 'FILEDELETE' Statement in your Macro...
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all 3 Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
pstefanov
Posts: 2
Joined: Tue May 19, 2015 4:14 am

Re: Extract and save a link from webpage

Post by pstefanov » Tue May 19, 2015 6:51 pm

Thanks a lot. Working Perfect. :P :P
chivracq
Posts: 8523
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Extract and save a link from webpage

Post by chivracq » Wed May 20, 2015 2:33 am

pstefanov wrote:Thanks a lot. Working Perfect. :P :P
Good to hear..., thanks for the Feedback... :D
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all 3 Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
provoquestosito
Posts: 62
Joined: Sat Jan 30, 2010 12:43 am

Re: Extract and save a link from webpage

Post by provoquestosito » Tue Jan 22, 2019 11:13 pm

Thanks to chivracq for this excellent and working script with eval in iim macro.
chivracq
Posts: 8523
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Extract and save a link from webpage

Post by chivracq » Wed Jan 23, 2019 1:00 pm

provoquestosito wrote:
Tue Jan 22, 2019 11:13 pm
Thanks to chivracq for this excellent and working script with eval in iim macro.
Thanks for the nice Feedback then... :D

Funny for me to see this old(er) Thread from nearly 4 years ago, ah-ah...! Hum, I was already doing some "interesting" Things with iMacros at that time, ah-ah...! But yep, the Technique still works and I still use it myself also indeed... 8)

The "fake" File can also be used to count the times that stg "goes wrong", and by adding some TimeStamp + any other relevant Data that can be collected throughout the Script, I also use this Technique to add some Logging Functionality to several of my Scripts..., and in a few even some "Double Logging": One Log with all Runs and all Data, and a separate Log containing only the Failures and Errors... :idea:

I especially use it for "longer" Scripts as I find it less "impacting" than the 'Profiler' (which slows Execution by about 30%), even if it requires a little bit of extra Coding, but I can include only the few Checks I want while the 'Profiler' Log-File quickly becomes a bit "unreadable" for Scripts that run regularly and that can be "a bit" long, like 5000-8000 Lines each in my case for my "regular" Scripts... 8)
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all 3 Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
provoquestosito
Posts: 62
Joined: Sat Jan 30, 2010 12:43 am

Re: Extract and save a link from webpage

Post by provoquestosito » Thu Jan 24, 2019 3:49 am

I found this post searching with google a solution to ""#EANF#" extraction and this is a excellent solution especially because working on iim macro.

I prefer iim because have some vantages like to run multiple macro simultaneusly moreover iim macro are to easy to edit and create respect javascript macros.

It is rare to find special functions for macros in iim format that support few commands, which is why this is an excellent solution.

I continue to read and study code of macros to know new tecnicques for automatization.

Have a nice day
Post Reply