Macro Doesn't Wait for Refresh to Complete

Support for iMacros. The iMacros software is the unique solution for automating every activity inside a web browser, for data extraction and web testing.
Forum rules
iMacros EOL - Attention!

The renewal maintenance has officially ended for Progress iMacros effective November 20, 2023 and all versions of iMacros are now considered EOL (End-of-Life). The iMacros products will no longer be supported by Progress (aside from customer license issues), and these forums will also no longer be moderated from the Progress side.

Thank you again for your business and support.

Sincerely,
The Progress Team

Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information: CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
Post Reply
aritzbastida
Posts: 4
Joined: Wed Apr 05, 2017 1:45 pm

Macro Doesn't Wait for Refresh to Complete

Post by aritzbastida » Wed Apr 05, 2017 9:17 pm

My use case is very simple: I have a table, with an array of buttons (labelled "OK"). I must press them one by one, wait for the dialog form to pop up, enter some text and then click on "Save" (a link that refreshes the page). I want to play it on a loop thanks to such an option in Chrome Add-On (Play macro repeatedly).

My script code is:

Code: Select all

TAG POS=1 TYPE=BUTTON FORM=ID:formulario ATTR=NAME:boton_OK
TAG POS=1 TYPE=TEXTAREA FORM=ID:formulario ATTR=ID:observaciones CONTENT=<SP>Validado
TAG POS=1 TYPE=A ATTR=TXT:Save
Problem: The last command actually clicks the "Save" link, but doesn´t wait for it to refresh the page. As a result, it will iterate through the loop, with unpredictable results. Normally, the net result is that only the first OK / Save buttons are triggered correctly.

I have googled and searched across the forum, but no solution I have found solves what I'm looking for. Of course, I could add a command such as WAIT SECONDS=5 at the end, but that will be inefficient and unsafe (the 90% of the time, the page is refreshed in under a second, but sometimes it takes 3-4 seconds). I have also tried with !TIMEOUT_PAGE and !TIMEOUT_STEP, but I perceived no difference.

Thanks for your help!!
chivracq
Posts: 10301
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Macro Doesn't Wait for Refresh to Complete

Post by chivracq » Wed Apr 05, 2017 11:01 pm

aritzbastida wrote:My use case is very simple: I have a table, with an array of buttons (labelled "OK"). I must press them one by one, wait for the dialog form to pop up, enter some text and then click on "Save" (a link that refreshes the page). I want to play it on a loop thanks to such an option in Chrome Add-On (Play macro repeatedly).

My script code is:

Code: Select all

TAG POS=1 TYPE=BUTTON FORM=ID:formulario ATTR=NAME:boton_OK
TAG POS=1 TYPE=TEXTAREA FORM=ID:formulario ATTR=ID:observaciones CONTENT=<SP>Validado
TAG POS=1 TYPE=A ATTR=TXT:Save
Problem: The last command actually clicks the "Save" link, but doesn´t wait for it to refresh the page. As a result, it will iterate through the loop, with unpredictable results. Normally, the net result is that only the first OK / Save buttons are triggered correctly.

I have googled and searched across the forum, but no solution I have found solves what I'm looking for. Of course, I could add a command such as WAIT SECONDS=5 at the end, but that will be inefficient and unsafe (the 90% of the time, the page is refreshed in under a second, but sometimes it takes 3-4 seconds). I have also tried with !TIMEOUT_PAGE and !TIMEOUT_STEP, but I perceived no difference.

Thanks for your help!!
CIM/FCIM...! :mrgreen:
I saw some "Chrome" somewhere in your Post, but mention your FCI (read my Sig...) for me to read, sorry... :roll:
Hum, and I don't see any URL mentioned, I hope your Qt is generic enough (and gives enough Info/Details) to expect a generic Answer as well... :idea:
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
aritzbastida
Posts: 4
Joined: Wed Apr 05, 2017 1:45 pm

Re: Macro Doesn't Wait for Refresh to Complete

Post by aritzbastida » Thu Apr 06, 2017 5:26 am

chivracq wrote:CIM/FCIM...! :mrgreen:
I saw some "Chrome" somewhere in your Post, but mention your FCI (read my Sig...) for me to read, sorry... :roll:
Hum, and I don't see any URL mentioned, I hope your Qt is generic enough (and gives enough Info/Details) to expect a generic Answer as well... :idea:
You are right, the question is generic, but the problem is generic as well, and that's why I wrote the post in the "General Support & Discussions" forum. :wink:

I didn't feel that the specific Chrome or iMacros extension version would make any difference, but here are the details:
* Chrome: 56.0.2924.87
* iMacros: 8.4.4
* OS: Windows 7

As for the URL, that's exactly the point: no explicit URL is involved in the script. The sequence is:
* I load a URL manually, with the table I mentioned
* I open iMacros extension, and Play the script above in a loop
* The third line triggers a click on a link, which saves the form information and refreshes that URL. It's nos an explicit URL load, but an implicit one.
* Probably because of that, the script won't wait the page to load and will continue to the next iteration in the loop. As the page has not been reloaded yet (still waiting for the HTTP result), for some reason, iMacros finds the TAG commands in the old page, and thus, the behavior of the script is wrong from this point on.

The URL is in a company's intranet, so I cannot show it for you to reproduce, but if necessary, I can try to reproduce the problem in some public website.
chivracq
Posts: 10301
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Macro Doesn't Wait for Refresh to Complete

Post by chivracq » Thu Apr 06, 2017 7:47 am

aritzbastida wrote:
chivracq wrote:CIM/FCIM...! :mrgreen:
I saw some "Chrome" somewhere in your Post, but mention your FCI (read my Sig...) for me to read, sorry... :roll:
Hum, and I don't see any URL mentioned, I hope your Qt is generic enough (and gives enough Info/Details) to expect a generic Answer as well... :idea:
You are right, the question is generic, but the problem is generic as well, and that's why I wrote the post in the "General Support & Discussions" forum. :wink:

I didn't feel that the specific Chrome or iMacros extension version would make any difference, but here are the details:

Code: Select all

* Chrome: 56.0.2924.87
* iMacros: 8.4.4
* OS: Windows 7
As for the URL, that's exactly the point: no explicit URL is involved in the script. The sequence is:
* I load a URL manually, with the table I mentioned
* I open iMacros extension, and Play the script above in a loop
* The third line triggers a click on a link, which saves the form information and refreshes that URL. It's nos an explicit URL load, but an implicit one.
* Probably because of that, the script won't wait the page to load and will continue to the next iteration in the loop. As the page has not been reloaded yet (still waiting for the HTTP result), for some reason, iMacros finds the TAG commands in the old page, and thus, the behavior of the script is wrong from this point on.

The URL is in a company's intranet, so I cannot show it for you to reproduce, but if necessary, I can try to reproduce the problem in some public website.
OK, good for FCI..., Required Info for me to read as many Commands are not implemented or behave differently for all Browsers/Versions, even if I always try to find a Solution that will work in all Browsers... FCI not mentioned... => I don't care, I don't read, simple, ah-ah...!

And the 'General' Sub-Forum is the correct Sub-Forum for your Qt, the 'FF' and 'CR' Sub-Forums are meant for Threads specific to only FF or CR, meaning that your Script works on all 3 other Browsers but fails only on FF or CR, or if you use a '.js' Script for FF as well as '.js' Scripts are only supported by iMacros for FF.

Well, your Pb is always specific to some specific URL or some specific Site or Page and iMacros Scripts are always tailored on a specific Site or Page especially when a Script doesn't behave like you would expect it to...

If I understand your Scenario correctly, the 'Save' triggers a Refresh of the Page but the next Loop doesn't wait for the Page to have fully loaded.

One possible Solution for Reliability (but not for Speed) could be to include the 'URL GOTO' at the beginning of your Script to force a "clean" Refresh of the Page (for which iMacros will wait for your '!TIMEOUT_PAGE' (Default) Setting).

As you loop your Script with always 'POS=1' (and not with 'POS={{!OOP}}'), "something" happens on the Page after every 'Save' with the 'OK' Buttons and their corresponding Input Field, either they disappear from the Page or the previous Input 'TEXTAREA' Field is still visible (with no 'OK' Button) and it is probably not of TYPE=TEXTAREA anymore...
=> If that is indeed the case, a possible Solution could be to start your Script by first tagging the "previous" "<SP>Validado" Field at 'POS={{!LOOP}}-1' with a "long" (= 5 sec or 10 sec) Value for '!TIMEOUT_STEP' before the "real" Click on the next 'OK' Button....
And you would exclude that 'POS={{!LOOP}}-1' Check Mechanism from the first Loop with 'EVAL()' or simply with '!ERRORIGNORE' to avoid the 'POS=0'.
Hum, or you could put that Check Mechanism at the end of the Macro after the 'Save' (=> based on 'POS={{!LOOP}}' then, and not on 'POS={{!LOOP}}-1' as you are still in the same Loop, like that you avoid the 'POS=0' Situation for the first Loop...) to make sure the 'Save' went OK before starting the next Loop. And if the Check doesn't go right, you can decide if the Macro will abort "naturally" because of your '!TIMEOUT_STEP' Setting or you could use a "Conditional PROMPT" or trigger a 'MacroError()' with 'EVAL()', or still decide to bypass that Error with '!ERRORIGNORE'...

Well, and if the "Rows" disappear one by one after each 'Save', I would have a few Solutions, but pfff..., that's the Pb with "generic" Solutions if I don't know how the Page behaves, I'll probably be typing that Solution for 1/4h for nothing, or still have to "cover" several Cases, if the Nb of Rows decreases or gets spanned on several Pages and the Nb of Rows remains constant on the first Page until there are less Rows than the Max Nb of Rows that can be displayed on one Page...
=> Describe a bit how your Page/Site "with no specific URL" behaves a bit exactly before I can think of a concrete Solution if that Behaviour is not the one I supposed in the first Solution, ah-ah...!

And yep, if you don't come out by yourself, you can always find some Public Site, but the Solution I will find will probably be tailored to that specific Site and won't work as-is on your own Site... But OK, good luck already with what I posted...
Last edited by chivracq on Thu Apr 06, 2017 8:30 am, edited 1 time in total.
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
aritzbastida
Posts: 4
Joined: Wed Apr 05, 2017 1:45 pm

Re: Macro Doesn't Wait for Refresh to Complete

Post by aritzbastida » Thu Apr 06, 2017 8:26 am

Wow, thanks for such a detailed answer!!

I forgot to mention that little detail: Yep, as you guessed, once you press the "Save" button, the form is submitted, the page refreshed and that item from the table will be gone. The table contains, let's say, I list of pending taks, which are submitted pressing the OK button, filling a textarea in the dialog box and finally clicking the "Save" link. As this a repetitive task (50+ items), I thought it could be automated with a macro.

I will slowly digest your answer, and try out some of your suggestions. Thank you, really.

However, I think that one thing stands out: iMacros has no specific command to wait for a TAG TYPE=A command to finish. One can control whether to follow the link or not (http://wiki.imacros.net/TAG#Following_links), but there is no option to wait for it. Conversely, URL GOTO has an associated !TIMEOUT_PAGE comand. Shouldn't be equivalent? (both of them trigger an HTTP GET)
chivracq
Posts: 10301
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Macro Doesn't Wait for Refresh to Complete

Post by chivracq » Thu Apr 06, 2017 9:39 am

aritzbastida wrote:Wow, thanks for such a detailed answer!!

I forgot to mention that little detail: Yep, as you guessed, once you press the "Save" button, the form is submitted, the page refreshed and that item from the table will be gone. The table contains, let's say, I list of pending taks, which are submitted pressing the OK button, filling a textarea in the dialog box and finally clicking the "Save" link. As this a repetitive task (50+ items), I thought it could be automated with a macro.

I will slowly digest your answer, and try out some of your suggestions. Thank you, really.
Ah OK, then that's the Case I didn't cover, I only provided a Solution for the Case the (previous) Row(s) would remain on the Page...
Then OK, but you don't mention if the 50 Rows are all displayed on one Page or if they are spanned on several Pages of 10-15-20-??? Rows for each Page...? Even if you have 268 Items, do they still all get displayed on one Page...?
Do you know in Advance (before launching your Script) exactly how many Items will will have to handle...? Is the Total Nb of Items displayed somewhere...?
Do New Items get added in Real Time to the "existing" 50 Items, maybe after you've already processed 15 Items, then suddenly it becomes 36 instead of 35...?

Implementation depends on how the Page is constructed... Can't you upload an HTML Saveas of a "typical" Page (zipped, Max 256Kb) to your Thread...? Or at least a Screenshot...? (But it would go much quicker if I could "play" with the Page myself, I guess I only need a few Minutes to write your Script
aritzbastida wrote:However, I think that one thing stands out: iMacros has no specific command to wait for a TAG TYPE=A command to finish. One can control whether to follow the link or not (http://wiki.imacros.net/TAG#Following_links), but there is no option to wait for it. Conversely, URL GOTO has an associated !TIMEOUT_PAGE comand. Shouldn't be equivalent? (both of them trigger an HTTP GET)
Well, that's the Case actually, I've never noticed anything "wrong" with how iMacros waits for 'TAG TYPE=A' Statements for 'Buttons', and the Sites where I run my own Macros have a lot of those...

iMacros for CR used to be pretty buggy prior to v8.4.4, this could be a "remain" from that time, ah-ah...! I use iMacros for FF (v8.8.2 or v8.9.7) and both behave correctly with 'TYPE=A' Statements for Buttons/Links... Even if hum..., I doubt it actually, v8.4.4 for CR has been available for nearly one year now, other Users would have reported it already, I would think... I would then rather tend to think that the Pb comes from your Site... Find a public Site where you can reproduce the Pb, then we can investigate it...
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
aritzbastida
Posts: 4
Joined: Wed Apr 05, 2017 1:45 pm

Re: Macro Doesn't Wait for Refresh to Complete

Post by aritzbastida » Thu Apr 06, 2017 2:10 pm

I just noticed that my "Save" link has some javascript inside; maybe that is confusing iMacros:

Code: Select all

<a href="javascript:saveValidation()">Save</a>
For your reference, please find attached the HTML Saveas in Pastebin: https://pastebin.com/1Ziek3iN

I tested it with in Firefox too (plugin version: 9.0.3), but experience the same problems. My guess is that iMacros has no way to know that some Javascript magic is happening under-the-hood, which first triggers an AJAX request to submit the form data and finally refreshes the page.
iimfun
Posts: 239
Joined: Tue Jul 19, 2016 1:06 pm

Re: Macro Doesn't Wait for Refresh to Complete

Post by iimfun » Fri Apr 07, 2017 7:50 am

@aritzbastida,

You can make the macro check the number of rows in the table and continue after it's got changed. Something like this

Code: Select all

SET !EXTRACT_TEST_POPUP NO
TAG XPATH="//table[@border='1']/tbody/tr[last()]/td[1]/center/b" EXTRACT=TXT

TAG POS=1 TYPE=BUTTON FORM=ID:formulario ATTR=NAME:boton_OK
TAG POS=1 TYPE=TEXTAREA FORM=ID:formulario ATTR=ID:observaciones CONTENT=<SP>Validado
TAG POS=1 TYPE=A ATTR=TXT:Save

SET !TIMEOUT_STEP 15
TAG XPATH="//table[@border='1']/tbody/tr[last()]/td[1]/center/b[not(text()='{{!EXTRACT}}')]"
SET !EXTRACT NULL
chivracq
Posts: 10301
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Macro Doesn't Wait for Refresh to Complete

Post by chivracq » Fri Apr 07, 2017 9:40 pm

aritzbastida wrote:I just noticed that my "Save" link has some javascript inside; maybe that is confusing iMacros:

Code: Select all

<a href="javascript:saveValidation()">Save</a>
For your reference, please find attached the HTML Saveas in Pastebin: https://pastebin.com/1Ziek3iN

I tested it with in Firefox too (plugin version: 9.0.3), but experience the same problems. My guess is that iMacros has no way to know that some Javascript magic is happening under-the-hood, which first triggers an AJAX request to submit the form data and finally refreshes the page.
iimfun wrote:@aritzbastida,

You can make the macro check the number of rows in the table and continue after it's got changed. Something like this

Code: Select all

SET !EXTRACT_TEST_POPUP NO
TAG XPATH="//table[@border='1']/tbody/tr[last()]/td[1]/center/b" EXTRACT=TXT

TAG POS=1 TYPE=BUTTON FORM=ID:formulario ATTR=NAME:boton_OK
TAG POS=1 TYPE=TEXTAREA FORM=ID:formulario ATTR=ID:observaciones CONTENT=<SP>Validado
TAG POS=1 TYPE=A ATTR=TXT:Save

SET !TIMEOUT_STEP 15
TAG XPATH="//table[@border='1']/tbody/tr[last()]/td[1]/center/b[not(text()='{{!EXTRACT}}')]"
SET !EXTRACT NULL
Grrr..., I hate this @iimfun, always on the Interesting Threads...! (Just joking..., ah-ah...!)

@OP, I had seen your previous Reply, didn't have the time to work on some Script, social Life and I'm a DJ and usually very busy in the "WE" from Th-Su, but @iimfun is good, "my" Sol will be for 60% similar to his, except that I don't use REGEX and XPATH, and I think my Sol will be easier to understand and to adapt for other Cases, ah-ah...! Still no time now to write it now, I'm already late for some Underground Rave Party and (one of) the DJ(s) is already late, oops...!

In a quick Nutshell if you understand what I mean, "my" Sol would be based on 'EXTRACT=HTM' on the 'TABLE' or containing 'DIV' + 'split('botom_ok')' (ah-ah...!, @iimfun spotted it as well...!) and some Check on the 'Array.length' Property (-1) of the 'split()'... :idea:
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
iimfun
Posts: 239
Joined: Tue Jul 19, 2016 1:06 pm

Re: Macro Doesn't Wait for Refresh to Complete

Post by iimfun » Fri Apr 14, 2017 7:37 am

chivracq wrote:Grrr..., I hate this @iimfun
It's okay, nobody is ideal...
chivracq
Posts: 10301
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Macro Doesn't Wait for Refresh to Complete

Post by chivracq » Fri Apr 14, 2017 9:31 am

iimfun wrote:
chivracq wrote:Grrr..., I hate this @iimfun
It's okay, nobody is ideal...
Ah-ah...!, LOL...! :twisted:

But, hum..., good that you bumped this Thread, I had already forgotten about it as many new Threads hit the Forum recently and I was the last Poster in the Thread, and hum..., already one week later and OP didn't follow up anymore to give some Feedback on your Solution..., grrr...! :shock:

And now I guess I will forget about it again as I'll be the last Poster again, and I post in so many Threads that I usually only check the Threads where I'm not the last Poster... But anyway, I won't go "writing" my own Solution if yours works and OP is happy with it, especially if OP doesn't follow up anymore on the Thread... :roll:
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
Dotty
Posts: 1
Joined: Fri Nov 17, 2017 9:20 am

Re: Macro Doesn't Wait for Refresh to Complete

Post by Dotty » Mon Nov 20, 2017 11:10 am

aritzbarstida wrote:
Thu Apr 06, 2017 8:26 am

However, I think PhenQ is the best and it stands out: iMacros has no specific command to wait for a TAG TYPE=A command to finish. One can control whether to follow the link or not (http://wiki.imacros.net/TAG#Following_links), but there is no option to wait for it. Conversely, URL GOTO has an associated !TIMEOUT_PAGE comand. Shouldn't be equivalent? (both of them trigger an HTTP GET)
Maybe you could try adding a time delay as well as wait for refresh. That's what I do sometimes.
Post Reply