Trimming a word off the extract value using EVAL + .substr?

Discussions and Tech Support related to website data extraction, screen scraping and data mining using iMacros.

Moderators: Community Moderators, iMacros Moderators

Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the Google search box (at the top of each forum page) to see if a similar problem or question has already been addressed. This will search the entire contents of the forums as well as the iMacros Wiki.
3. We can respond much faster to your posts if you include the following information:

CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST

Answering your own posts (e.g. attempting to "bump" your topic) drops your topic from the list of unanswered threads, so it may actually receive less views.

Trimming a word off the extract value using EVAL + .substr?

by xCloudx on Sun Jan 21, 2018 11:34 pm

iMacros for Firefox 9.0.3//Firefox 49.0.2//Windows 10 Pro [Eng]

So I'm going a little nuts here. I haven't used iMacros in a long time (last post on this account was 2011) and I seem to have forgotten a lot. But I had a problem automating some link clicking and I thought, hey that old iMacros extension might work! So here I am.

So my end goal for my script is to click on several hyperlinked downloads (.torrent files) to automate the downloading process for me.

First problem:
The amount of torrents on the page change, depending on what specific page you're looking at. Some pages on the site may have 7 torrents, others have 2.
Luckily the site lists the amount of torrents on a page in a text attribute (literal text is 'Torrents x').
So first, I need to extract the 'x'. Through quite a bit of searching, it seems the best way to do this is using EVAL + javascript to manipulate the extraction. I settled on using substr because I only need the last value for the extraction and the word Torrents is always the same amount of characters. This is what I have so far:
Code: Select all
SET !EXTRACT_TEST_POPUP NO
TAG POS=1 TYPE=LI ATTR=TXT:Torrents:<SP>* EXTRACT=TXT
SET !VAR1 EVAL("'{{!EXTRACT}}'.substr(10,1);")
PROMPT {{!var1}}

Now, I know that I've got some wrong syntax in there, I'm just not fully understanding where it's at/what is wrong. I know it's something with the EVAL because if I just use SET !VAR1 {{!EXTRACT}} instead of the EVAL in the same code, I get the prompt: Torrents: x. So everything's working, all I want is the correct syntax for trimming Torrents:<SP> off and having VAR1=x.
--------------------------
My second problem (getting ahead of myself a bit here but better to ask now I guess):
This is the line of code to click on the download links. Basically there are multiple instances of this text on the page, all corresponding to links to different torrents. Each torrent link looks like this:
[DownLoad]
So to find those links and click on them I use:
Code: Select all
TAG POS=1 TYPE=A ATTR=TXT:DownLoad
TAG POS=2 TYPE=A ATTR=TXT:DownLoad
TAG POS=3 TYPE=A ATTR=TXT:DownLoad
etc...

What I need is to use the VAR1 from my first problem above and have it define the maximum amount of TAG POS to use. Basically if there's 23 torrents on the site, I need it to run:
Code: Select all
TAG POS=1 TYPE=A ATTR=TXT:DownLoad
...
TAG POS=23 TYPE=A ATTR=TXT:DownLoad
.
How would I go about doing that? I understand I can use TAG POS={{!var1}} to find the final DownLoad but I'm not sure how to use that to count up or how to design it so it stops the code after a certain amount.

Hopefully I explained myself clearly enough, if you need anymore information feel free to ask me to clarify. There's probably a much simpler way of doing this but I'm obviously clueless as to what it is.
xCloudx
 
Posts: 12
Joined: Tue Mar 06, 2007 7:02 pm

Re: Trimming a word off the extract value using EVAL + .subs

by chivracq on Mon Jan 22, 2018 3:17 am

xCloudx wrote:
Code: Select all
iMacros for Firefox 9.0.3//Firefox 49.0.2//Windows 10 Pro [Eng]


So I'm going a little nuts here. I haven't used iMacros in a long time (last post on this account was 2011) and I seem to have forgotten a lot. But I had a problem automating some link clicking and I thought, hey that old iMacros extension might work! So here I am.

So my end goal for my script is to click on several hyperlinked downloads (.torrent files) to automate the downloading process for me.

First problem:
The amount of torrents on the page change, depending on what specific page you're looking at. Some pages on the site may have 7 torrents, others have 2.
Luckily the site lists the amount of torrents on a page in a text attribute (literal text is 'Torrents x').
So first, I need to extract the 'x'. Through quite a bit of searching, it seems the best way to do this is using EVAL + javascript to manipulate the extraction. I settled on using substr because I only need the last value for the extraction and the word Torrents is always the same amount of characters. This is what I have so far:
Code: Select all
SET !EXTRACT_TEST_POPUP NO
TAG POS=1 TYPE=LI ATTR=TXT:Torrents:<SP>* EXTRACT=TXT
SET !VAR1 EVAL("'{{!EXTRACT}}'.substr(10,1);")
PROMPT {{!var1}}

Now, I know that I've got some wrong syntax in there, I'm just not fully understanding where it's at/what is wrong. I know it's something with the EVAL because if I just use SET !VAR1 {{!EXTRACT}} instead of the EVAL in the same code, I get the prompt: Torrents: x. So everything's working, all I want is the correct syntax for trimming Torrents:<SP> off and having VAR1=x.
--------------------------
My second problem (getting ahead of myself a bit here but better to ask now I guess):
This is the line of code to click on the download links. Basically there are multiple instances of this text on the page, all corresponding to links to different torrents. Each torrent link looks like this:
[DownLoad]
So to find those links and click on them I use:
Code: Select all
TAG POS=1 TYPE=A ATTR=TXT:DownLoad
TAG POS=2 TYPE=A ATTR=TXT:DownLoad
TAG POS=3 TYPE=A ATTR=TXT:DownLoad
etc...

What I need is to use the VAR1 from my first problem above and have it define the maximum amount of TAG POS to use. Basically if there's 23 torrents on the site, I need it to run:
Code: Select all
TAG POS=1 TYPE=A ATTR=TXT:DownLoad
...
TAG POS=23 TYPE=A ATTR=TXT:DownLoad
.
How would I go about doing that? I understand I can use TAG POS={{!var1}} to find the final DownLoad but I'm not sure how to use that to count up or how to design it so it stops the code after a certain amount.

Hopefully I explained myself clearly enough, if you need anymore information feel free to ask me to clarify. There's probably a much simpler way of doing this but I'm obviously clueless as to what it is.

You are indeed one of the oldest "Dinosaurs" on this Forum, ah-ah...! You registered in 2007, woaw...! :wink:

OK, concerning your Qt, the easiest (a bit dirty!) Solution would simply be to let your Script loop say 50 or 100 times on:
Code: Select all
TAG POS={{!LOOP}} TYPE=A ATTR=TXT:DownLoad
... and your Script will simply automatically abort at Loop=24 because there won't be any 'POS=24' 'Download' Link anymore. ('!ERRORIGNORE' needs to be/remain disabled...)

If you want the Abort to be a little bit "cleaner", you can force it yourself with 'EVAL()' + 'MacroError()' at Loop=23, either based on that 'n=23' that you'll have extracted, or on 'Loop+1' to check for each Loop if there is still a next Download Link.

Other Solution is indeed to extract this 'n=23' Nb of Torrents, pass it to a main '.js' Script that will then reuse that Value to loop exactly 23 times a Submacro handling the Download part.

OK now, to isolate this "23" from your "Torrents: 23" extracted String, yep, your 'EVAL()' Statement with 'substr()' looks a bit OK to me even if I use a different Syntax for myself, except that you will get only "2" and not "23" as you've restricted the Length of the Substring to 1 Char only.

Try this one, using "my" Syntax:
Code: Select all
TAG POS=1 TYPE=LI ATTR=TXT:Torrents:<SP>* EXTRACT=TXT
SET !VAR1 EVAL("var s='{{!EXTRACT}}'; var z=s.substr(10); z;")
PROMPT EXTRACT:<SP>_{{!EXTRACT}}_<BR>VAR1:<SP>_{{!VAR1}}_


And rather than using 'substr()', I find 'split()' more flexible and powerful...:
Code: Select all
TAG POS=1 TYPE=LI ATTR=TXT:Torrents:<SP>* EXTRACT=TXT
SET !VAR1 EVAL("var s='{{!EXTRACT}}'; var y,z; y=s.split(' '); z=y[1]; z;")
PROMPT EXTRACT:<SP>_{{!EXTRACT}}_<BR>VAR1:<SP>_{{!VAR1}}_

(Not tested...)
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all 3 Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
chivracq
 
Posts: 6691
Joined: Sat Apr 13, 2013 6:07 am
Location: Amsterdam (NL)


Return to Data Extraction and Web Screen Scraping

Who is online

Users browsing this forum: No registered users and 4 guests

-->