Page 1 of 1

Macro Doesn't Wait for Refresh to Complete

Posted: Wed Apr 05, 2017 9:17 pm
by aritzbastida
My use case is very simple: I have a table, with an array of buttons (labelled "OK"). I must press them one by one, wait for the dialog form to pop up, enter some text and then click on "Save" (a link that refreshes the page). I want to play it on a loop thanks to such an option in Chrome Add-On (Play macro repeatedly).

My script code is:

Code: Select all

TAG POS=1 TYPE=BUTTON FORM=ID:formulario ATTR=NAME:boton_OK
TAG POS=1 TYPE=TEXTAREA FORM=ID:formulario ATTR=ID:observaciones CONTENT=<SP>Validado
TAG POS=1 TYPE=A ATTR=TXT:Save
Problem: The last command actually clicks the "Save" link, but doesn´t wait for it to refresh the page. As a result, it will iterate through the loop, with unpredictable results. Normally, the net result is that only the first OK / Save buttons are triggered correctly.

I have googled and searched across the forum, but no solution I have found solves what I'm looking for. Of course, I could add a command such as WAIT SECONDS=5 at the end, but that will be inefficient and unsafe (the 90% of the time, the page is refreshed in under a second, but sometimes it takes 3-4 seconds). I have also tried with !TIMEOUT_PAGE and !TIMEOUT_STEP, but I perceived no difference.

Thanks for your help!!

Re: Macro Doesn't Wait for Refresh to Complete

Posted: Wed Apr 05, 2017 11:01 pm
by chivracq
aritzbastida wrote:My use case is very simple: I have a table, with an array of buttons (labelled "OK"). I must press them one by one, wait for the dialog form to pop up, enter some text and then click on "Save" (a link that refreshes the page). I want to play it on a loop thanks to such an option in Chrome Add-On (Play macro repeatedly).

My script code is:

Code: Select all

TAG POS=1 TYPE=BUTTON FORM=ID:formulario ATTR=NAME:boton_OK
TAG POS=1 TYPE=TEXTAREA FORM=ID:formulario ATTR=ID:observaciones CONTENT=<SP>Validado
TAG POS=1 TYPE=A ATTR=TXT:Save
Problem: The last command actually clicks the "Save" link, but doesn´t wait for it to refresh the page. As a result, it will iterate through the loop, with unpredictable results. Normally, the net result is that only the first OK / Save buttons are triggered correctly.

I have googled and searched across the forum, but no solution I have found solves what I'm looking for. Of course, I could add a command such as WAIT SECONDS=5 at the end, but that will be inefficient and unsafe (the 90% of the time, the page is refreshed in under a second, but sometimes it takes 3-4 seconds). I have also tried with !TIMEOUT_PAGE and !TIMEOUT_STEP, but I perceived no difference.

Thanks for your help!!
CIM/FCIM...! :mrgreen:
I saw some "Chrome" somewhere in your Post, but mention your FCI (read my Sig...) for me to read, sorry... :roll:
Hum, and I don't see any URL mentioned, I hope your Qt is generic enough (and gives enough Info/Details) to expect a generic Answer as well... :idea:

Re: Macro Doesn't Wait for Refresh to Complete

Posted: Thu Apr 06, 2017 5:26 am
by aritzbastida
chivracq wrote:CIM/FCIM...! :mrgreen:
I saw some "Chrome" somewhere in your Post, but mention your FCI (read my Sig...) for me to read, sorry... :roll:
Hum, and I don't see any URL mentioned, I hope your Qt is generic enough (and gives enough Info/Details) to expect a generic Answer as well... :idea:
You are right, the question is generic, but the problem is generic as well, and that's why I wrote the post in the "General Support & Discussions" forum. :wink:

I didn't feel that the specific Chrome or iMacros extension version would make any difference, but here are the details:
* Chrome: 56.0.2924.87
* iMacros: 8.4.4
* OS: Windows 7

As for the URL, that's exactly the point: no explicit URL is involved in the script. The sequence is:
* I load a URL manually, with the table I mentioned
* I open iMacros extension, and Play the script above in a loop
* The third line triggers a click on a link, which saves the form information and refreshes that URL. It's nos an explicit URL load, but an implicit one.
* Probably because of that, the script won't wait the page to load and will continue to the next iteration in the loop. As the page has not been reloaded yet (still waiting for the HTTP result), for some reason, iMacros finds the TAG commands in the old page, and thus, the behavior of the script is wrong from this point on.

The URL is in a company's intranet, so I cannot show it for you to reproduce, but if necessary, I can try to reproduce the problem in some public website.

Re: Macro Doesn't Wait for Refresh to Complete

Posted: Thu Apr 06, 2017 7:47 am
by chivracq
aritzbastida wrote:
chivracq wrote:CIM/FCIM...! :mrgreen:
I saw some "Chrome" somewhere in your Post, but mention your FCI (read my Sig...) for me to read, sorry... :roll:
Hum, and I don't see any URL mentioned, I hope your Qt is generic enough (and gives enough Info/Details) to expect a generic Answer as well... :idea:
You are right, the question is generic, but the problem is generic as well, and that's why I wrote the post in the "General Support & Discussions" forum. :wink:

I didn't feel that the specific Chrome or iMacros extension version would make any difference, but here are the details:

Code: Select all

* Chrome: 56.0.2924.87
* iMacros: 8.4.4
* OS: Windows 7
As for the URL, that's exactly the point: no explicit URL is involved in the script. The sequence is:
* I load a URL manually, with the table I mentioned
* I open iMacros extension, and Play the script above in a loop
* The third line triggers a click on a link, which saves the form information and refreshes that URL. It's nos an explicit URL load, but an implicit one.
* Probably because of that, the script won't wait the page to load and will continue to the next iteration in the loop. As the page has not been reloaded yet (still waiting for the HTTP result), for some reason, iMacros finds the TAG commands in the old page, and thus, the behavior of the script is wrong from this point on.

The URL is in a company's intranet, so I cannot show it for you to reproduce, but if necessary, I can try to reproduce the problem in some public website.
OK, good for FCI..., Required Info for me to read as many Commands are not implemented or behave differently for all Browsers/Versions, even if I always try to find a Solution that will work in all Browsers... FCI not mentioned... => I don't care, I don't read, simple, ah-ah...!

And the 'General' Sub-Forum is the correct Sub-Forum for your Qt, the 'FF' and 'CR' Sub-Forums are meant for Threads specific to only FF or CR, meaning that your Script works on all 3 other Browsers but fails only on FF or CR, or if you use a '.js' Script for FF as well as '.js' Scripts are only supported by iMacros for FF.

Well, your Pb is always specific to some specific URL or some specific Site or Page and iMacros Scripts are always tailored on a specific Site or Page especially when a Script doesn't behave like you would expect it to...

If I understand your Scenario correctly, the 'Save' triggers a Refresh of the Page but the next Loop doesn't wait for the Page to have fully loaded.

One possible Solution for Reliability (but not for Speed) could be to include the 'URL GOTO' at the beginning of your Script to force a "clean" Refresh of the Page (for which iMacros will wait for your '!TIMEOUT_PAGE' (Default) Setting).

As you loop your Script with always 'POS=1' (and not with 'POS={{!OOP}}'), "something" happens on the Page after every 'Save' with the 'OK' Buttons and their corresponding Input Field, either they disappear from the Page or the previous Input 'TEXTAREA' Field is still visible (with no 'OK' Button) and it is probably not of TYPE=TEXTAREA anymore...
=> If that is indeed the case, a possible Solution could be to start your Script by first tagging the "previous" "<SP>Validado" Field at 'POS={{!LOOP}}-1' with a "long" (= 5 sec or 10 sec) Value for '!TIMEOUT_STEP' before the "real" Click on the next 'OK' Button....
And you would exclude that 'POS={{!LOOP}}-1' Check Mechanism from the first Loop with 'EVAL()' or simply with '!ERRORIGNORE' to avoid the 'POS=0'.
Hum, or you could put that Check Mechanism at the end of the Macro after the 'Save' (=> based on 'POS={{!LOOP}}' then, and not on 'POS={{!LOOP}}-1' as you are still in the same Loop, like that you avoid the 'POS=0' Situation for the first Loop...) to make sure the 'Save' went OK before starting the next Loop. And if the Check doesn't go right, you can decide if the Macro will abort "naturally" because of your '!TIMEOUT_STEP' Setting or you could use a "Conditional PROMPT" or trigger a 'MacroError()' with 'EVAL()', or still decide to bypass that Error with '!ERRORIGNORE'...

Well, and if the "Rows" disappear one by one after each 'Save', I would have a few Solutions, but pfff..., that's the Pb with "generic" Solutions if I don't know how the Page behaves, I'll probably be typing that Solution for 1/4h for nothing, or still have to "cover" several Cases, if the Nb of Rows decreases or gets spanned on several Pages and the Nb of Rows remains constant on the first Page until there are less Rows than the Max Nb of Rows that can be displayed on one Page...
=> Describe a bit how your Page/Site "with no specific URL" behaves a bit exactly before I can think of a concrete Solution if that Behaviour is not the one I supposed in the first Solution, ah-ah...!

And yep, if you don't come out by yourself, you can always find some Public Site, but the Solution I will find will probably be tailored to that specific Site and won't work as-is on your own Site... But OK, good luck already with what I posted...

Re: Macro Doesn't Wait for Refresh to Complete

Posted: Thu Apr 06, 2017 8:26 am
by aritzbastida
Wow, thanks for such a detailed answer!!

I forgot to mention that little detail: Yep, as you guessed, once you press the "Save" button, the form is submitted, the page refreshed and that item from the table will be gone. The table contains, let's say, I list of pending taks, which are submitted pressing the OK button, filling a textarea in the dialog box and finally clicking the "Save" link. As this a repetitive task (50+ items), I thought it could be automated with a macro.

I will slowly digest your answer, and try out some of your suggestions. Thank you, really.

However, I think that one thing stands out: iMacros has no specific command to wait for a TAG TYPE=A command to finish. One can control whether to follow the link or not (http://wiki.imacros.net/TAG#Following_links), but there is no option to wait for it. Conversely, URL GOTO has an associated !TIMEOUT_PAGE comand. Shouldn't be equivalent? (both of them trigger an HTTP GET)

Re: Macro Doesn't Wait for Refresh to Complete

Posted: Thu Apr 06, 2017 9:39 am
by chivracq
aritzbastida wrote:Wow, thanks for such a detailed answer!!

I forgot to mention that little detail: Yep, as you guessed, once you press the "Save" button, the form is submitted, the page refreshed and that item from the table will be gone. The table contains, let's say, I list of pending taks, which are submitted pressing the OK button, filling a textarea in the dialog box and finally clicking the "Save" link. As this a repetitive task (50+ items), I thought it could be automated with a macro.

I will slowly digest your answer, and try out some of your suggestions. Thank you, really.
Ah OK, then that's the Case I didn't cover, I only provided a Solution for the Case the (previous) Row(s) would remain on the Page...
Then OK, but you don't mention if the 50 Rows are all displayed on one Page or if they are spanned on several Pages of 10-15-20-??? Rows for each Page...? Even if you have 268 Items, do they still all get displayed on one Page...?
Do you know in Advance (before launching your Script) exactly how many Items will will have to handle...? Is the Total Nb of Items displayed somewhere...?
Do New Items get added in Real Time to the "existing" 50 Items, maybe after you've already processed 15 Items, then suddenly it becomes 36 instead of 35...?

Implementation depends on how the Page is constructed... Can't you upload an HTML Saveas of a "typical" Page (zipped, Max 256Kb) to your Thread...? Or at least a Screenshot...? (But it would go much quicker if I could "play" with the Page myself, I guess I only need a few Minutes to write your Script
aritzbastida wrote:However, I think that one thing stands out: iMacros has no specific command to wait for a TAG TYPE=A command to finish. One can control whether to follow the link or not (http://wiki.imacros.net/TAG#Following_links), but there is no option to wait for it. Conversely, URL GOTO has an associated !TIMEOUT_PAGE comand. Shouldn't be equivalent? (both of them trigger an HTTP GET)
Well, that's the Case actually, I've never noticed anything "wrong" with how iMacros waits for 'TAG TYPE=A' Statements for 'Buttons', and the Sites where I run my own Macros have a lot of those...

iMacros for CR used to be pretty buggy prior to v8.4.4, this could be a "remain" from that time, ah-ah...! I use iMacros for FF (v8.8.2 or v8.9.7) and both behave correctly with 'TYPE=A' Statements for Buttons/Links... Even if hum..., I doubt it actually, v8.4.4 for CR has been available for nearly one year now, other Users would have reported it already, I would think... I would then rather tend to think that the Pb comes from your Site... Find a public Site where you can reproduce the Pb, then we can investigate it...

Re: Macro Doesn't Wait for Refresh to Complete

Posted: Thu Apr 06, 2017 2:10 pm
by aritzbastida
I just noticed that my "Save" link has some javascript inside; maybe that is confusing iMacros:

Code: Select all

<a href="javascript:saveValidation()">Save</a>
For your reference, please find attached the HTML Saveas in Pastebin: https://pastebin.com/1Ziek3iN

I tested it with in Firefox too (plugin version: 9.0.3), but experience the same problems. My guess is that iMacros has no way to know that some Javascript magic is happening under-the-hood, which first triggers an AJAX request to submit the form data and finally refreshes the page.

Re: Macro Doesn't Wait for Refresh to Complete

Posted: Fri Apr 07, 2017 7:50 am
by iimfun
@aritzbastida,

You can make the macro check the number of rows in the table and continue after it's got changed. Something like this

Code: Select all

SET !EXTRACT_TEST_POPUP NO
TAG XPATH="//table[@border='1']/tbody/tr[last()]/td[1]/center/b" EXTRACT=TXT

TAG POS=1 TYPE=BUTTON FORM=ID:formulario ATTR=NAME:boton_OK
TAG POS=1 TYPE=TEXTAREA FORM=ID:formulario ATTR=ID:observaciones CONTENT=<SP>Validado
TAG POS=1 TYPE=A ATTR=TXT:Save

SET !TIMEOUT_STEP 15
TAG XPATH="//table[@border='1']/tbody/tr[last()]/td[1]/center/b[not(text()='{{!EXTRACT}}')]"
SET !EXTRACT NULL

Re: Macro Doesn't Wait for Refresh to Complete

Posted: Fri Apr 07, 2017 9:40 pm
by chivracq
aritzbastida wrote:I just noticed that my "Save" link has some javascript inside; maybe that is confusing iMacros:

Code: Select all

<a href="javascript:saveValidation()">Save</a>
For your reference, please find attached the HTML Saveas in Pastebin: https://pastebin.com/1Ziek3iN

I tested it with in Firefox too (plugin version: 9.0.3), but experience the same problems. My guess is that iMacros has no way to know that some Javascript magic is happening under-the-hood, which first triggers an AJAX request to submit the form data and finally refreshes the page.
iimfun wrote:@aritzbastida,

You can make the macro check the number of rows in the table and continue after it's got changed. Something like this

Code: Select all

SET !EXTRACT_TEST_POPUP NO
TAG XPATH="//table[@border='1']/tbody/tr[last()]/td[1]/center/b" EXTRACT=TXT

TAG POS=1 TYPE=BUTTON FORM=ID:formulario ATTR=NAME:boton_OK
TAG POS=1 TYPE=TEXTAREA FORM=ID:formulario ATTR=ID:observaciones CONTENT=<SP>Validado
TAG POS=1 TYPE=A ATTR=TXT:Save

SET !TIMEOUT_STEP 15
TAG XPATH="//table[@border='1']/tbody/tr[last()]/td[1]/center/b[not(text()='{{!EXTRACT}}')]"
SET !EXTRACT NULL
Grrr..., I hate this @iimfun, always on the Interesting Threads...! (Just joking..., ah-ah...!)

@OP, I had seen your previous Reply, didn't have the time to work on some Script, social Life and I'm a DJ and usually very busy in the "WE" from Th-Su, but @iimfun is good, "my" Sol will be for 60% similar to his, except that I don't use REGEX and XPATH, and I think my Sol will be easier to understand and to adapt for other Cases, ah-ah...! Still no time now to write it now, I'm already late for some Underground Rave Party and (one of) the DJ(s) is already late, oops...!

In a quick Nutshell if you understand what I mean, "my" Sol would be based on 'EXTRACT=HTM' on the 'TABLE' or containing 'DIV' + 'split('botom_ok')' (ah-ah...!, @iimfun spotted it as well...!) and some Check on the 'Array.length' Property (-1) of the 'split()'... :idea:

Re: Macro Doesn't Wait for Refresh to Complete

Posted: Fri Apr 14, 2017 7:37 am
by iimfun
chivracq wrote:Grrr..., I hate this @iimfun
It's okay, nobody is ideal...

Re: Macro Doesn't Wait for Refresh to Complete

Posted: Fri Apr 14, 2017 9:31 am
by chivracq
iimfun wrote:
chivracq wrote:Grrr..., I hate this @iimfun
It's okay, nobody is ideal...
Ah-ah...!, LOL...! :twisted:

But, hum..., good that you bumped this Thread, I had already forgotten about it as many new Threads hit the Forum recently and I was the last Poster in the Thread, and hum..., already one week later and OP didn't follow up anymore to give some Feedback on your Solution..., grrr...! :shock:

And now I guess I will forget about it again as I'll be the last Poster again, and I post in so many Threads that I usually only check the Threads where I'm not the last Poster... But anyway, I won't go "writing" my own Solution if yours works and OP is happy with it, especially if OP doesn't follow up anymore on the Thread... :roll:

Re: Macro Doesn't Wait for Refresh to Complete

Posted: Mon Nov 20, 2017 11:10 am
by Dotty
aritzbarstida wrote:
Thu Apr 06, 2017 8:26 am

However, I think PhenQ is the best and it stands out: iMacros has no specific command to wait for a TAG TYPE=A command to finish. One can control whether to follow the link or not (http://wiki.imacros.net/TAG#Following_links), but there is no option to wait for it. Conversely, URL GOTO has an associated !TIMEOUT_PAGE comand. Shouldn't be equivalent? (both of them trigger an HTTP GET)
Maybe you could try adding a time delay as well as wait for refresh. That's what I do sometimes.