Has something changed with Firefox Imacros Ondownload?

Discussions and Tech Support related to the iMacros Firefox Add-on, including the built-in Javascript scripting interface for playing .js files.
Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information: CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
Post Reply
pvallen
Posts: 9
Joined: Wed Apr 24, 2019 2:25 am

Has something changed with Firefox Imacros Ondownload?

Post by pvallen » Wed Apr 24, 2019 3:28 am

Hi there,

I am using Firefox Quantum 66.0.3, and am using Imacros for Firefox 10.0.2.1450, which I have paid for. I created a Macro recently that was working perfectly, and now I can't successfully download an image to the folder I decide, with a filename I decide.

Our supplier offers images for their products. My macro is successful up to getting to this URL:

https://aentmedia.azureedge.net/prod-im ... e=67628070

but when I try and download it, it's saving it as the original filename, and to the folder as determined in Firefox settings. Here is my macro so far (which was working recently)

VERSION BUILD=10021450
TAB T=1
SET !REPLAYSPEED FAST
SET !EXTRACT_TEST_POPUP yes
SET !ERRORIGNORE YES
SET !EXTRACT NULL
'loop through list of UPC numbers
TAG POS={{!LOOP}} TYPE=TD ATTR=TXT:* EXTRACT=TXT
SET !VAR1 {{!EXTRACT}}
SET !EXTRACT NULL
TAB T=2
'go to the supplier's page for that UPC
URL GOTO=http://webami.aent.com/{{!VAR1}}
'extract the title of the UPC, which is the ALT text for the image
TAG POS=1 TYPE=h1 ATTR=TXT:* EXTRACT=TXT
SET !VAR3 {{!EXTRACT}}
SET !EXTRACT NULL
'extract the url of the image
TAG POS=1 TYPE=IMG ATTR=ALT:{{!VAR3}}&&SRC:* EXTRACT=HREF
'if the image doesn't exist, save my reset page as the URL
SET !VAR2 EVAL("var s='{{!EXTRACT}}'; var x; if(s==\"#EANF#\") {x=\"d3d71ba2asa5oz.cloudfront.net/62000501/images/lildot.jpg\";} else {x=s;} x;")
SET !EXTRACT NULL
TAB T=3
'open the image only up in a new tab. see example url above
URL GOTO={{!VAR2}}
TAG POS=1 TYPE=IMG ATTR=HREF:https://* CONTENT=EVENT:SAVEPICTUREAS
ONDOWNLOAD FOLDER=C:\Users\Paul\Dropbox\IMAGES_MACRO FILE={{!VAR1}}__1.jpg WAIT=YES
SET !TIMEOUT_STEP 2
'End by going to a "reset" page to stop the chance of the same image getting saved as multiple filenames
URL GOTO=https://d3d71ba2asa5oz.cloudfront.net/6 ... lildot.jpg
WAIT SECONDS=2

I have tried setting Firefox to Always ask you where to save files, and also tried setting Firefox to Save files to. Unfortunately I'm not getting a file named UPC__1.jpg into the folder I specified.

I really need to get this back up and running. No idea if Firefox has updated (although I have it set to notify me and not update automatically) or whether something has happened with iMacros. Help! Please!

Thanks,

Paul
chivracq
Posts: 8693
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Has something changed with Firefox Imacros Ondownload?

Post by chivracq » Wed Apr 24, 2019 4:10 am

pvallen wrote:
Wed Apr 24, 2019 3:28 am
Hi there,

I am using Firefox Quantum 66.0.3, and am using Imacros for Firefox 10.0.2.1450, which I have paid for. I created a Macro recently that was working perfectly, and now I can't successfully download an image to the folder I decide, with a filename I decide.

Our supplier offers images for their products. My macro is successful up to getting to this URL:

https://aentmedia.azureedge.net/prod-im ... e=67628070

but when I try and download it, it's saving it as the original filename, and to the folder as determined in Firefox settings. Here is my macro so far (which was working recently)

Code: Select all

VERSION BUILD=10021450
TAB T=1
SET !REPLAYSPEED FAST
SET !EXTRACT_TEST_POPUP yes
SET !ERRORIGNORE YES
SET !EXTRACT NULL
'loop through list of UPC numbers
TAG POS={{!LOOP}} TYPE=TD ATTR=TXT:* EXTRACT=TXT
SET !VAR1 {{!EXTRACT}}
SET !EXTRACT NULL
TAB T=2
'go to the supplier's page for that UPC
URL GOTO=http://webami.aent.com/{{!VAR1}}
'extract the title of the UPC, which is the ALT text for the image
TAG POS=1 TYPE=h1 ATTR=TXT:* EXTRACT=TXT
SET !VAR3 {{!EXTRACT}}
SET !EXTRACT NULL
'extract the url of the image
TAG POS=1 TYPE=IMG ATTR=ALT:{{!VAR3}}&&SRC:* EXTRACT=HREF
'if the image doesn't exist, save my reset page as the URL
SET !VAR2 EVAL("var s='{{!EXTRACT}}'; var x; if(s==\"#EANF#\") {x=\"d3d71ba2asa5oz.cloudfront.net/62000501/images/lildot.jpg\";} else {x=s;} x;")
SET !EXTRACT NULL
TAB T=3
'open the image only up in a new tab. see example url above
URL GOTO={{!VAR2}}
TAG POS=1 TYPE=IMG ATTR=HREF:https://* CONTENT=EVENT:SAVEPICTUREAS
ONDOWNLOAD FOLDER=C:\Users\Paul\Dropbox\IMAGES_MACRO FILE={{!VAR1}}__1.jpg WAIT=YES
SET !TIMEOUT_STEP 2
'End by going to a "reset" page to stop the chance of the same image getting saved as multiple filenames
URL GOTO=https://d3d71ba2asa5oz.cloudfront.net/62000501/images/lildot.jpg
WAIT SECONDS=2
I have tried setting Firefox to Always ask you where to save files, and also tried setting Firefox to Save files to. Unfortunately I'm not getting a file named UPC__1.jpg into the folder I specified.

I really need to get this back up and running. No idea if Firefox has updated (although I have it set to notify me and not update automatically) or whether something has happened with iMacros. Help! Please!

Thanks,

Paul

Yeah, but the 'ONDOWNLOAD' Command needs to be placed before the Statement that will trigger the Download, not after...!:

Code: Select all

URL GOTO={{!VAR2}}
ONDOWNLOAD FOLDER=C:\Users\Paul\Dropbox\IMAGES_MACRO FILE={{!VAR1}}__1.jpg WAIT=YES
TAG POS=1 TYPE=IMG ATTR=HREF:https://* CONTENT=EVENT:SAVEPICTUREAS
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE').
- I don't even read the Qt if that (required) Info is not mentioned...!
- Script & URL help a lot for more "educated" Help...
pvallen
Posts: 9
Joined: Wed Apr 24, 2019 2:25 am

Re: Has something changed with Firefox Imacros Ondownload?

Post by pvallen » Wed Apr 24, 2019 5:00 am

Hi chivracq,

Thanks for the reply. I should have mentioned that I tried those commands around both ways. I just changed it again so that the Ondownload command is forst. Same result, but iMacros is giving the following error:

Error: node_moveTo => Illegal characters in path., line: 21

Any other suggestions to try?

Thanks,
Paul
chivracq
Posts: 8693
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Has something changed with Firefox Imacros Ondownload?

Post by chivracq » Wed Apr 24, 2019 5:37 am

pvallen wrote:
Wed Apr 24, 2019 5:00 am
Hi chivracq,

Thanks for the reply. I should have mentioned that I tried those commands around both ways. I just changed it again so that the Ondownload command is forst. Same result, but iMacros is giving the following error:

Error: node_moveTo => Illegal characters in path., line: 21

Any other suggestions to try?

Thanks,
Paul

Ah...!, there is "Hope" then, ah-ah...! :wink:

Well, you use a '!VAR1' in your Filename, check the Content of that Var as it comes from:

Code: Select all

TAG POS={{!LOOP}} TYPE=TD ATTR=TXT:* EXTRACT=TXT
SET !VAR1 {{!EXTRACT}}
PROMPT VAR1:<SP>_{{!VAR1}}_
... to make sure for example you don't have any (soft) Return(s) or Tab(s) in the Extract... that would need to be removed with:

Code: Select all

TAG POS={{!LOOP}} TYPE=TD ATTR=TXT:* EXTRACT=TXT
SET !VAR1 EVAL("var s='{{!EXTRACT}}'; var z=s.trim(); z;")
PROMPT EXTRACT:<SP>_{{!EXTRACT}}_<BR><BR>VAR1:<SP>_{{!VAR1}}_
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE').
- I don't even read the Qt if that (required) Info is not mentioned...!
- Script & URL help a lot for more "educated" Help...
pvallen
Posts: 9
Joined: Wed Apr 24, 2019 2:25 am

Re: Has something changed with Firefox Imacros Ondownload?

Post by pvallen » Wed Apr 24, 2019 5:44 am

Thankyou so much chivracq. Looks like I had some whacky tabs in my list of UPCs to grab. I changed that file, and it's all running perfectly again! Thanks!

One last question. While hunting through the iMacros wiki for stuff, I thought there might be a way to save the image from the href I extract, without going to 3rd tab, loading it and then saving it. Is it possible to do save it at the point of getting the image URL earlier in the macro? Any other changes you can suggest to make it run quicker/smoother?

Thanks again,
Paul
chivracq
Posts: 8693
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Has something changed with Firefox Imacros Ondownload?

Post by chivracq » Wed Apr 24, 2019 6:25 am

pvallen wrote:
Wed Apr 24, 2019 5:44 am
Thankyou so much chivracq. Looks like I had some whacky tabs in my list of UPCs to grab. I changed that file, and it's all running perfectly again! Thanks!

One last question. While hunting through the iMacros wiki for stuff, I thought there might be a way to save the image from the href I extract, without going to 3rd tab, loading it and then saving it. Is it possible to do save it at the point of getting the image URL earlier in the macro? Any other changes you can suggest to make it run quicker/smoother?

Thanks again,
Paul

OK, good to hear...! :D
And as you can see, 'PROMPT' is always your "best Friend" to follow your Vars and to debug your Script...! :idea:

About your "last Qt", ah-ah...!, funny, but I "nearly" mentioned in my first Reply that I was finding your Use of a 3rd Tab only to (re)load that Image to save it, a bit cumbersome, ah-ah...! Oh, and hum, your 'SAVEPICTUREAS' Command is also deprecated btw, replaced by 'SAVEITEM'...

But yep, you already have the URL and the Image loaded a bit earlier when you are on TAB_2:

Code: Select all

SET !EXTRACT NULL
'extract the url of the image
TAG POS=1 TYPE=IMG ATTR=ALT:{{!VAR3}}&&SRC:* EXTRACT=HREF
=> There you could already download it with:

Code: Select all

ONDOWNLOAD FOLDER=C:\Users\Paul\Dropbox\IMAGES_MACRO FILE={{!VAR1}}__1.jpg WAIT=YES
TAG POS=1 TYPE=IMG ATTR=ALT:{{!VAR3}}&&SRC:* CONTENT=EVENT:SAVEITEM
And if the "first" Image is just a Thumbnail with a Link leading to a "second" Image with a bigger Resolution, you could use:

Code: Select all

ONDOWNLOAD FOLDER=C:\Users\Paul\Dropbox\IMAGES_MACRO FILE={{!VAR1}}__1.jpg WAIT=YES
TAG POS=1 TYPE=IMG ATTR=ALT:{{!VAR3}}&&SRC:* CONTENT=EVENT:SAVETARGETAS
And if you want to include the Mechanism to only save the Image if the URL is found, you could use stg like:

Code: Select all

SET !ERRORIGNORE YES
SET !TIMEOUT_STEP 0
SET !EXTRACT NULL
TAG POS=1 TYPE=IMG ATTR=ALT:{{!VAR3}}&&SRC:* EXTRACT=HREF
'Check if the Image exists:
SET Image_Exist EVAL("var s='{{!EXTRACT}}'; var z; if(s!='#EANF#'){z=1;} else{z=0;}; z;")

ONDOWNLOAD FOLDER=C:\Users\Paul\Dropbox\IMAGES_MACRO FILE={{!VAR1}}__1.jpg WAIT=YES
TAG POS={{Image_Exist}} TYPE=IMG ATTR=ALT:{{!VAR3}}&&SRC:* CONTENT=EVENT:SAVEITEM
... And the 'TAG POS=0' won't do anything if the Image doesn't exist... :wink:
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE').
- I don't even read the Qt if that (required) Info is not mentioned...!
- Script & URL help a lot for more "educated" Help...
pvallen
Posts: 9
Joined: Wed Apr 24, 2019 2:25 am

Re: Has something changed with Firefox Imacros Ondownload?

Post by pvallen » Wed Apr 24, 2019 7:16 am

Great! Thanks so much. This is what I have now:

VERSION BUILD=10021450
TAB T=1
SET !REPLAYSPEED FAST
SET !EXTRACT_TEST_POPUP NO
SET !ERRORIGNORE YES
SET !EXTRACT NULL
TAG POS={{!LOOP}} TYPE=TD ATTR=TXT:* EXTRACT=TXT
SET !VAR1 {{!EXTRACT}}
SET !EXTRACT NULL
TAB T=2
URL GOTO=http://webami.aent.com/{{!VAR1}}
TAG POS=1 TYPE=h1 ATTR=TXT:* EXTRACT=TXT
SET !VAR3 {{!EXTRACT}}
SET !TIMEOUT_STEP 0
SET !EXTRACT NULL
TAG POS=1 TYPE=IMG ATTR=ALT:{{!VAR3}}&&SRC:* EXTRACT=HREF
'Check if the Image exists:
SET Image_Exist EVAL("var s='{{!EXTRACT}}'; var z; if(s!='#EANF#'){z=1;} else{z=0;}; z;")
ONDOWNLOAD FOLDER=C:\Users\Paul\Dropbox\IMAGES_MACRO FILE={{!VAR1}}__1.jpg WAIT=YES
TAG POS={{Image_Exist}} TYPE=IMG ATTR=ALT:{{!VAR3}}&&SRC:* CONTENT=EVENT:SAVETARGETAS

I just looped it and it ran successfully for about 18 times, and then it just stopped on the last line. I waited for about 70 seconds as the imacro little timer kept going, but it didn't get past the last line. I had to stop the macro.

I can see in the destination folder, as the macro is running, that it saves the image under the original filename, and then my filename is saved. Looks like instead of the image it got stuck on, there is a file called 3637084.jpg.part

I just tried starting it again, but starting it from the UPC it died on last time. It sat on the last line for about 10 seconds, and then successfully saved the image and moved on. Unfortunately it got to another image about 5 down the list and has been "Loading file" for 150+ seconds. I can't really trust the macro to run by itself for hundreds/thousands of images until I know why it's stalling. Any ideas?
chivracq
Posts: 8693
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Has something changed with Firefox Imacros Ondownload?

Post by chivracq » Wed Apr 24, 2019 7:51 am

pvallen wrote:
Wed Apr 24, 2019 7:16 am
Great! Thanks so much. This is what I have now:

Code: Select all

VERSION BUILD=10021450
TAB T=1
SET !REPLAYSPEED FAST
SET !EXTRACT_TEST_POPUP NO
SET !ERRORIGNORE YES
SET !EXTRACT NULL
TAG POS={{!LOOP}} TYPE=TD ATTR=TXT:* EXTRACT=TXT
SET !VAR1 {{!EXTRACT}}
SET !EXTRACT NULL
TAB T=2
URL GOTO=http://webami.aent.com/{{!VAR1}}
TAG POS=1 TYPE=h1 ATTR=TXT:* EXTRACT=TXT
SET !VAR3 {{!EXTRACT}}
SET !TIMEOUT_STEP 0
SET !EXTRACT NULL
TAG POS=1 TYPE=IMG ATTR=ALT:{{!VAR3}}&&SRC:* EXTRACT=HREF
'Check if the Image exists:
SET Image_Exist EVAL("var s='{{!EXTRACT}}'; var z; if(s!='#EANF#'){z=1;} else{z=0;}; z;")
ONDOWNLOAD FOLDER=C:\Users\Paul\Dropbox\IMAGES_MACRO FILE={{!VAR1}}__1.jpg WAIT=YES
TAG POS={{Image_Exist}} TYPE=IMG ATTR=ALT:{{!VAR3}}&&SRC:* CONTENT=EVENT:SAVETARGETAS
I just looped it and it ran successfully for about 18 times, and then it just stopped on the last line. I waited for about 70 seconds as the imacro little timer kept going, but it didn't get past the last line. I had to stop the macro.

I can see in the destination folder, as the macro is running, that it saves the image under the original filename, and then my filename is saved. Looks like instead of the image it got stuck on, there is a file called 3637084.jpg.part

I just tried starting it again, but starting it from the UPC it died on last time. It sat on the last line for about 10 seconds, and then successfully saved the image and moved on. Unfortunately it got to another image about 5 down the list and has been "Loading file" for 150+ seconds. I can't really trust the macro to run by itself for hundreds/thousands of images until I know why it's stalling. Any ideas?

Ah OK, so you have to use 'SAVETARGETAS' from the TAB_2, meaning a "second" new Image needs to be fetched/downloaded and not the "first" Image is saved, which iMacros with 'SAVEITEM' (or your previous 'SAVEPICTUREAS' does from the Browser Cache and doesn't download it again...

Then hum, I suspect that your Script runs/loops too quickly and is "clogging" the Download Manager of the Browser and you might need to slow down your Script a little bit...
What could play a role also is your Internet Connection and/or if the Site is a bit "busy"...

What you could do is already to shorten the '!TIMEOUT_DOWNLOAD' which by Default = 300 Sec on v10.0.2 for FF. :idea:

And you could implement a "Fail-Safe" Mechanism, using 'STOPWATCH'/'!STOPWATCHTIME' and/or '!NOW', and if the '!TIMEOUT_DOWNLOAD' Value has been reached, using 'EVAL()' to conditionally switch to TAB_3 to still reload the Picture from its URL like you were doing before to try again to save it from this way... :idea:
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE').
- I don't even read the Qt if that (required) Info is not mentioned...!
- Script & URL help a lot for more "educated" Help...
pvallen
Posts: 9
Joined: Wed Apr 24, 2019 2:25 am

Re: Has something changed with Firefox Imacros Ondownload?

Post by pvallen » Wed Apr 24, 2019 11:25 am

So I changed the CONTENT=EVENT to SAVEITEM, but it still looks like it's either downloading a file with the original filename, and then another filen with my filename, and then removing the original, OR it's downloading the original filename, then renaming it to the new filename. It definitely looks like sometimes I can see both files for a split second in File Explorer though

I changed the !REPLAYSPEED to MEDIUM instead of FAST at the start of the macro, but the whole thing feels like it runs quite slow now, and I actually got an error I'd not seen before:

BadParameter: expected POS=<number> or POS=R<number>where <number> is a non-zero integer as parameter 1, line: 21

Do you think I should change between MEDIUM and FAST throughout the macro. Are there certain steps I should be running in MEDIUM and others that are ok at FAST speed?

I did a speed test on my internet and it seems pretty normal for us, at around 42Mbps. Not sure about how busy the site is, but it's probably dependant on the time of day, as I'm in Australia, and the site is American.

I also changed the !TIMEOUT_DOWNLOAD to be 20. Is that way too short?

Here's the macro I have currently:

VERSION BUILD=10021450
TAB T=1
SET !REPLAYSPEED MEDIUM
SET !EXTRACT_TEST_POPUP NO
SET !ERRORIGNORE YES
SET !EXTRACT NULL
TAG POS={{!LOOP}} TYPE=TD ATTR=TXT:* EXTRACT=TXT
SET !VAR1 {{!EXTRACT}}
SET !EXTRACT NULL
TAB T=2
URL GOTO=http://webami.aent.com/{{!VAR1}}
TAG POS=1 TYPE=h1 ATTR=TXT:* EXTRACT=TXT
SET !VAR3 {{!EXTRACT}}
SET !TIMEOUT_STEP 0
SET !EXTRACT NULL
TAG POS=1 TYPE=IMG ATTR=ALT:{{!VAR3}}&&SRC:* EXTRACT=HREF
'Check if the Image exists:
SET Image_Exist EVAL("var s='{{!EXTRACT}}'; var z; if(s!='#EANF#'){z=1;} else{z=0;}; z;")
SET !TIMEOUT_DOWNLOAD 20
ONDOWNLOAD FOLDER=C:\Users\Paul\Dropbox\IMAGES_MACRO FILE={{!VAR1}}__1.jpg WAIT=YES
TAG POS={{Image_Exist}} TYPE=IMG ATTR=ALT:{{!VAR3}}&&SRC:* CONTENT=EVENT:SAVEITEM

The Fail-Safe is a little over my head at the moment!
pvallen
Posts: 9
Joined: Wed Apr 24, 2019 2:25 am

Re: Has something changed with Firefox Imacros Ondownload?

Post by pvallen » Wed Apr 24, 2019 11:37 am

Hi chivracq,

I just looked further into why that error came up:
BadParameter: expected POS=<number> or POS=R<number>where <number> is a non-zero integer as parameter 1, line: 21

Looks like the ALT text on that UPC didn't match the Title. The Title (in h1, line 12) was Great "Baby" Blues (Various Artists), but the ALT was Great &quot;Baby&quot; Blues (Various Artists)

I just looked at the HTML of the page. Looks like for every UPC, all these (smaller versions of the) images actually call the same class:

<div class="aec-cover col-md-4 col-sm-4 col-xs-12">
<img alt="Great &amp;quot;Baby&amp;quot; Blues (Various Artists)" src="https://aentcdn.azureedge.net/graphics/ ... e=67628070">
</div>

Is there a more elegant way to save the image at the URL above, without all the extra steps of extracting the title, and finding an image on the page with that Title as the ALT text?

Thanks again for your help so far!

Cheers,
Paul
pvallen
Posts: 9
Joined: Wed Apr 24, 2019 2:25 am

Re: Has something changed with Firefox Imacros Ondownload?

Post by pvallen » Sun Apr 28, 2019 9:08 am

Hi chivracq,

I started writing it again with this new idea of grabbing the image from within the div where the class="aec-cover col-md-4 col-sm-4 col-xs-12"

I just don't know the syntax for line 12 to be able to do that:

VERSION BUILD=10021450
TAB T=1
SET !REPLAYSPEED FAST
SET !EXTRACT_TEST_POPUP YES
SET !ERRORIGNORE YES
SET !EXTRACT NULL
TAG POS={{!LOOP}} TYPE=TD ATTR=TXT:* EXTRACT=TXT
SET !VAR1 {{!EXTRACT}}
SET !EXTRACT NULL
TAB T=2
URL GOTO=http://webami.aent.com/{{!VAR1}}
TAG POS=1 TYPE=IMG ATTR=class:aec-cover<SP>col-md-4<SP>col-sm-4<SP>col-xs-12 EXTRACT=HREF
'Check if the Image exists:
SET Image_Exist EVAL("var s='{{!EXTRACT}}'; var z; if(s!='#EANF#'){z=1;} else{z=0;}; z;")
SET !TIMEOUT_DOWNLOAD 20
SET !REPLAYSPEED MEDIUM
ONDOWNLOAD FOLDER=C:\Users\Paul\Dropbox\IMAGES_MACRO FILE={{!VAR1}}__1.jpg WAIT=YES
TAG POS={{Image_Exist}} TYPE=IMG ATTR=ALT:*&&SRC:*&&CLASS:aec-cover<SP>col-md-4<SP>col-sm-4<SP>col-xs-12 CONTENT=EVENT:SAVEITEM

Do you know to pull the image from within that div with that class? Thanks again for all your help with this!

Cheers,
Paul
pvallen
Posts: 9
Joined: Wed Apr 24, 2019 2:25 am

Re: Has something changed with Firefox Imacros Ondownload?

Post by pvallen » Thu May 02, 2019 3:26 pm

Hi chivracq,

I looked into the relative positioning, and also looked at the website code a bit more. Looks like the div before the div containing the image is always the same...so, in line 12 I'm finding the div before, then in line 13, I'm using relative positioning to go to the next img after the div before:

1 VERSION BUILD=10021450
2 TAB T=1
3 SET !REPLAYSPEED FAST
4 SET !EXTRACT_TEST_POPUP NO
5 SET !ERRORIGNORE YES
6 SET !EXTRACT NULL
7 TAG POS={{!LOOP}} TYPE=TD ATTR=TXT:* EXTRACT=TXT
8 SET !VAR1 {{!EXTRACT}}
9 SET !EXTRACT NULL
10 TAB T=2
11 URL GOTO=http://webami.aent.com/{{!VAR1}}
12 TAG POS=1 TYPE=DIV ATTR=class:main-title<SP>col-xs-12<SP>hidden-lg<SP>hidden-sm<SP>hidden-md
13 TAG POS=R1 TYPE=IMG ATTR=* EXTRACT=HREF
14 SET !VAR2 {{!EXTRACT}}
15 SET !EXTRACT NULL
16 SET Image_Exist EVAL("var s='{{!VAR2}}'; var z; if(s!='#EANF#'){z=1;} else{z=0;}; z;")
17 SET !TIMEOUT_DOWNLOAD 20
18 SET !REPLAYSPEED MEDIUM
19 ONDOWNLOAD FOLDER=C:\Users\Paul\Dropbox\IMAGES_MACRO FILE={{!VAR1}}__1.jpg WAIT=YES
20 TAG POS={{Image_Exist}} TYPE=IMG ATTR=ALT:*&&SRC:{{!VAR2}} CONTENT=EVENT:SAVEITEM
21 WAIT SECONDS=3

I'm occassionaly getting an error that says:
RuntimeError: Download timeout, line: 20

I'm assuming it's still running too fast for the download manager of Firefox? Thanks in advance for any help you're able to give!
chivracq
Posts: 8693
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Has something changed with Firefox Imacros Ondownload?

Post by chivracq » Thu May 02, 2019 6:08 pm

pvallen wrote:
Thu May 02, 2019 3:26 pm
Hi chivracq,

I looked into the relative positioning, and also looked at the website code a bit more. Looks like the div before the div containing the image is always the same...so, in line 12 I'm finding the div before, then in line 13, I'm using relative positioning to go to the next img after the div before:

Code: Select all

1	VERSION BUILD=10021450
2	TAB T=1
3	SET !REPLAYSPEED FAST
4	SET !EXTRACT_TEST_POPUP NO
5	SET !ERRORIGNORE YES
6	SET !EXTRACT NULL
7	TAG POS={{!LOOP}} TYPE=TD ATTR=TXT:* EXTRACT=TXT
8	SET !VAR1 {{!EXTRACT}}
9	SET !EXTRACT NULL
10	TAB T=2
11	URL GOTO=http://webami.aent.com/{{!VAR1}}
12	TAG POS=1 TYPE=DIV ATTR=class:main-title<SP>col-xs-12<SP>hidden-lg<SP>hidden-sm<SP>hidden-md
13	TAG POS=R1 TYPE=IMG ATTR=* EXTRACT=HREF
14	SET !VAR2 {{!EXTRACT}}
15	SET !EXTRACT NULL
16	SET Image_Exist EVAL("var s='{{!VAR2}}'; var z; if(s!='#EANF#'){z=1;} else{z=0;}; z;")
17	SET !TIMEOUT_DOWNLOAD 20
18	SET !REPLAYSPEED MEDIUM
19	ONDOWNLOAD FOLDER=C:\Users\Paul\Dropbox\IMAGES_MACRO FILE={{!VAR1}}__1.jpg WAIT=YES
20	TAG POS={{Image_Exist}} TYPE=IMG ATTR=ALT:*&&SRC:{{!VAR2}} CONTENT=EVENT:SAVEITEM
21	WAIT SECONDS=3
I'm occassionaly getting an error that says:
RuntimeError: Download timeout, line: 20

I'm assuming it's still running too fast for the download manager of Firefox? Thanks in advance for any help you're able to give!
Yeah, well, it's a bit difficult (for me) to follow what your Script is doing as you didn't post the Start-URL from where you start your Script, then some later 'URL GOTO' uses a Var that comes from some Extract and I don't know what that Var contains and what the URL is..., and it looks like the whole Site is behind Login&Password anyway, I had tried but could only see/access the Homepage...
Then you mention some 'DIV'-'DIV'-'DIV'-etc and some Image(s), and everything is "sample"/"example"/"particularClass" or stg generic, but I would need to see by myself how the HTML Structure of the Page is organized, and I cannot "reconstruct" it mentally from some mini-Excerpts from the HTML Source... :roll:

I had understood from your parallel or related Thread on SOF that the Image you want is located inside some 'DIV' Element that you can tag easily, but 'POS=R1' wouldn't work, that's why I mentioned you had to use 'DOUBLE Relative Positioning', hum..., or 'XPATH' also, I thought afterwards, that could be another Option, "my" 'Double R-Pos' Method is actually my "Workaround" to avoid using 'XPATH' that I find too "complicated" ah-ah... :oops:
So, if your 'POS=R1' works, that means the Image is not inside the 'DIV' but after, and at the same or higher Level, or you managed to find some other 'DIV'' which is at the same Level in the HTML Hierarchy of the Page like the Image...

For the 'Download Timeout' Error, that would come from the Site I would think, or the Browser is "over-busy" with downloading/renaming/moving/copying the Files, even if 'SAVEITEM' normally saves the Picture from the Browser Cache, so the Image doesn't need to be downloaded again normally...
And you've included some 'WAIT' Statement in your Script and you control '!REPLAYSPEED', so that should be OK, I would think...

But hum, I see that you are using '!ERRORIGNORE' in your Script, are you saying that you still get the 'Download Timeout' RuntimeError even with '!ERRORIGNORE'...!? If that's the case, I would think that's not "normal"... :shock:
Well, I don't do any Downloads in my own Scripts, so I've never really dug deep into that Functionality and done some "deep" Testing for myself... And v10.0.x for FF and CR now use the separate FIO-Module for File Access and Download and Path and Rename etc..., so it could also mean that this Version doesn't behave exactly anymore like the previous Versions, I don't know... :oops:
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE').
- I don't even read the Qt if that (required) Info is not mentioned...!
- Script & URL help a lot for more "educated" Help...
pvallen
Posts: 9
Joined: Wed Apr 24, 2019 2:25 am

Re: Has something changed with Firefox Imacros Ondownload?

Post by pvallen » Fri May 03, 2019 8:11 am

Hi chivracq,

Thanks again for the reply ;P

Sorry I drip fed the info before. I didn't think all the names of the attributes were the focus, but I totally understand that for you it's easier to see the whole lot.

The page that is open on tab 1 is something like this (but with a lot more lines):

<TABLE BORDER="1">
<TR><TD>819478023479</TD></TR>
<TR><TD>819478023486</TD></TR>
<TR><TD>819873018117</TD></TR>
<TR><TD>819873018643</TD></TR>
<TR><TD>819873018650</TD></TR>
</TABLE>

It's a list of the UPCs (barcodes) of the new CDs and DVDs available to us. LIne 7 and 8 are looping through the list of the UPCs, each time setting the UPC to be !VAR1

Line 11 goes to our supplier's website. It requires a login and password, but this is the code around where the cover image of the CD is:

<div class="main-title col-xs-12 hidden-lg hidden-sm hidden-md">
...</div>
<div class="aec-cover col-md-4 col-sm-4 col-xs-12">
<img alt="Social Cues" src="https://aentmedia.azureedge.net/prod-im ... e=68910270">
</div>

So with your help on the other forum at SOF about relative positioning, I'm now anchoring to the previous div (with the class starting with main-title), and am then finding the next image. All of that seems to be working fine. Occassionally the UPC that is searched for no longer exists in the suppliers system, so those divs above do not exist on those results. I need to work out another EVAL line in there to check that. That's tonight's mission!

Yeah I am seeing the errors in the iMacros window, down the bottom. And yes I have ERRORIGNORE set to YES. Not sure why that's happening.

Thanks again for your help. I really appreciate it. You don't do any Airtasker style paid jobs do you? I have a few of these lilttle macros I need to make. I'd much rather pay you!
chivracq
Posts: 8693
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Has something changed with Firefox Imacros Ondownload?

Post by chivracq » Fri May 03, 2019 5:35 pm

pvallen wrote:
Fri May 03, 2019 8:11 am
Hi chivracq,

Thanks again for the reply ;P

Sorry I drip fed the info before. I didn't think all the names of the attributes were the focus, but I totally understand that for you it's easier to see the whole lot.

The page that is open on tab 1 is something like this (but with a lot more lines):

Code: Select all

<TABLE BORDER="1">
<TR><TD>819478023479</TD></TR>
<TR><TD>819478023486</TD></TR>
<TR><TD>819873018117</TD></TR>
<TR><TD>819873018643</TD></TR>
<TR><TD>819873018650</TD></TR>
</TABLE>
It's a list of the UPCs (barcodes) of the new CDs and DVDs available to us. LIne 7 and 8 are looping through the list of the UPCs, each time setting the UPC to be !VAR1

Line 11 goes to our supplier's website. It requires a login and password, but this is the code around where the cover image of the CD is:

Code: Select all

<div class="main-title col-xs-12 hidden-lg hidden-sm hidden-md">
...</div>
<div class="aec-cover col-md-4 col-sm-4 col-xs-12">
<img alt="Social Cues" src="https://aentmedia.azureedge.net/prod-img/500/71/3635171.jpg?ae=68910270">
</div>
So with your help on the other forum at SOF about relative positioning, I'm now anchoring to the previous div (with the class starting with main-title), and am then finding the next image. All of that seems to be working fine. Occassionally the UPC that is searched for no longer exists in the suppliers system, so those divs above do not exist on those results. I need to work out another EVAL line in there to check that. That's tonight's mission!

Yeah I am seeing the errors in the iMacros window, down the bottom. And yes I have ERRORIGNORE set to YES. Not sure why that's happening.

Thanks again for your help. I really appreciate it. You don't do any Airtasker style paid jobs do you? I have a few of these lilttle macros I need to make. I'd much rather pay you!
OK, perfect, makes sense with all "Explanation", ah-ah...!

Then yep, "single/standard" 'Relative Positioning' (=> 'POS=R1') can be used with the 'main-title' 'DIV' while "Double" 'Relative Positioning' (=> 'POS=R-1' + 'POS=R1' or 'POS=R1' + 'POS=R-1' (the Direction sometimes makes a "Difference", ah-ah...!)) would have been needed with the 'aec-cover' 'DIV' as 'Anchor'. 8)

When using a 'DIV' as Anchor, 'Relative Positioning' cannot see "inside" the 'DIV', it "only" starts looking after (or before with 'POS=R-1') the 'DIV'. I'm not sure if it's "by Design", and I never got any "Statement" about that Behaviour from TechSup, or if the Programmer "forgot" about it when he implemented the 'TAG' Command, or I guess their Solution in that Case was to use 'XPATH', but that's often a "Problem" with (containing) 'DIV' or 'SPAN' Elements...
And that's where you then need to use 'Double R-POS', ah-ah...! Would be nice if the 'TAG' Command could support an extra Parameter where that Behaviour could be toggled ON/OFF... Hum, I think I will add it to my List of EnhReq's, ah-ah...!

For the missing UPC's Functionality, OK, I let you "think" a little bit about it, I use it a lot myself in my Scripts (I call it the 'PLC-Method' or 'PLC-Check', and "PLC" stands for...?), as I often have a very slow and/or unreliable Internet Connection with regular Scripts of 5000-8000-13000 Lines running every day that I cannot monitor all the time, so I have implemented a lot of "Robustness" and extra-Checks in my Scripts, to make sure for example that when an '#EANF#' gets returned, and is used for some further Conditional Logic, it really comes from the Field being empty or the HTML Element missing, and not that I lost my Internet Connection...

Yeah I am seeing the errors in the iMacros window, down the bottom. And yes I have ERRORIGNORE set to YES. Not sure why that's happening.
"I am seeing", OK, but is the RuntimeError "only" displayed (for Info) in the Side Panel but the Script continues to run...?, or does the Script get aborted...?
The first Behaviour would be "normal", if you want to dismiss all RuntimeErrors, you can also check the CB in the iMacros Options about "ignoring Errors about legacy and not supported Commands" or stg like that, that I had managed to get implemented during the v10.0.2 for FF Beta-Phase, for Backward Compatibility...

About 'Airtasker' mini-Jobs, had never heard of this Site, OK, is Australian I see, hum, like your IP, ah-ah...! Eh beeeeh..., yeah, why not...! I have some PayPal Account for that Purpose, ah-ah...!, even if I'm always happy to help Users for Free on the Forum, I'm pretty anti-commercial actually... :wink:
Your Script(s) look(s) interesting indeed, the kind that I like, ah-ah...! And that would get you "started" a bit "quicker" indeed as I would be able to implement a few "useful" and "Best Practice" Mechanisms in a few of your Scripts that you then afterwards could easily reuse yourself... 8)
Hum, you in Australia, and me in NL (Europe), Time Zone Difference will be Fun, ah-ah...!! :wink:
You can use [My_Username_on_the_Forum] for 'Skype' and/or '[]@gmail.com' for Off-Forum Communication through 'Skype' (Chat) and/or E-mail...
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE').
- I don't even read the Qt if that (required) Info is not mentioned...!
- Script & URL help a lot for more "educated" Help...
Post Reply