How to detect page loading completion after leaving the tab

Discussions and Tech Support related to the iMacros Firefox Add-on, including the built-in Javascript scripting interface for playing .js files.
Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information: CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
Post Reply
serbeer
Posts: 44
Joined: Fri Sep 11, 2015 5:36 am

How to detect page loading completion after leaving the tab

Post by serbeer » Tue Jun 21, 2016 10:19 pm

Hello,
this is my first post as I've always been able to find my questions already answered on this forum, but not this time...

I am running FF v47 with Imacros plugin v8.9.7 on Windows 7 64-bit system.
I am trying to speed up filling out 3 to 18 forms, with each open in its own Firefox tab by .iim script.

I currently use .iim script to
a)open with TAB OPEN, then T=1, then URL GOTO=http:\\... commands
b)waiting for page to finish loading based on standard SET !TIMEOUT_PAGE 60
c)fillout the form with number of TAGs
d)submit without waiting for results using

Code: Select all

SET !TIMEOUT_PAGE 1
SET !ERRORIGNORE YES
TAG POS=1 TYPE=BUTTON FORM=NAME:NoFormName ATTR=ID:search-button 
Then I repeat for tabs #2 and so on. That works perfectly fine, but is not very fast as it leads to 3-18 waits for each page load, one-by-one.

I realized that I could speed up this process considerably, by factor of 1.5 in my experiments, if I only pre-load each page by opening them one by one first to let them load in parallel breaking out before they finish loading, then fill them out one by one in the second "rotation" of each tab. Something like

Code: Select all

SET !TIMEOUT_PAGE 1
SET !ERRORIGNORE YES
a)open with TAB OPEN, then T=1, then URL GOTO=http:\\URL1.com
b)open with TAB OPEN, then T=2, then URL GOTO=http:\\URL2.com
' ...Then I repeat for each of 3-18 tabs, pre-fetching, then go back to fill out the forms
c)T=1
   SET !ERRORIGNORE NO
   SET !TIMEOUT_PAGE 60
   'fill out form in tab 1 here
d) submit without waiting for results with 
    SET !TIMEOUT_PAGE 1
    SET !ERRORIGNORE YES
   TAG POS=1 TYPE=BUTTON FORM=NAME:NoFormName ATTR=ID:search-button
e) T=2
   SET !ERRORIGNORE NO
   SET !TIMEOUT_PAGE 60
   'fill out form in tab 2 here
f) submit without waiting for results with 
    SET !TIMEOUT_PAGE 1
    SET !ERRORIGNORE YES
   TAG POS=1 TYPE=BUTTON FORM=NAME:NoFormName2 ATTR=ID:search-button-2
' ...Then I repeat for each of 3-18 tabs, filling out the forms on pages already hopefully loaded
As I said, that indeed results in much time savings, however, I now have problems with the fact that sometimes I came back to tab #1 (after opening all other tabs and initiating load in them) before the webpage in tab #1 had a chance to load fully. In such cases, my script frequently errors out trying to fill out form elements.

I tried introducing a few seconds delay after coming back to tab #1 with WAIT statement, but this is not efficient as sometimes I lose time on wait when the page is already loaded, and sometimes the delay proves insufficient. I tried to use
SET !TIMEOUT_STEP 10
TAG ... EXTRACT=TXT
but cannot find good element on the page to wait for, it is not obvious what loads in which order, and the pages are very complicated taking tens of seconds sometimes to load everything. Neither do I know of a way to find out what could be a reliable flag in native page full of CSS--nothing I found shows order of elements loading for a webpage. Perhaps you know of a tool to see loading order? Firebug only shows network communications timeline, not loading timeline.

On the other hand, Imacros has no problem detecting page loading completion correctly if I simply wait for URL GOTO=http:\\... step completion, as long as I am not breaking out using
SET !TIMEOUT_PAGE 1
SET !ERRORIGNORE YES
to pre-fetch other pages.

So, perhaps someone clever on this forum knows a way to make it detect/wait for page loading completion after coming back to the tab I broke out of before it had a chance to load fully? I am not sure what mechanism it normally uses to detect page load completion, but is there a way to "re-activate" it after coming back?

Or use some other trick/method of page loading completion detection after coming back to the tab?


Simply re-setting
SET !ERRORIGNORE NO
SET !TIMEOUT_PAGE 60
after re-entering the tab with load in progress has no effect, does not wait for page load completion.

I am already considering really tricky approaches, eg using Greasemonkey to inject some "flag" element into page upon loading completion which then I would be able to detect with Imacros TAG, eg:

Code: Select all

//Greasemonkey script
$(document).ready(function()
        {
            //add some flag element to the page here, perhaps in a DIV, like "Finished" link
        }
    );
but that approach is really complicated and will degrade performance since Greasemonkey will be running in each page. Is there something simpler in existence?

Hope someone here may have an advice for me. Thank you very much!
chivracq
Posts: 8871
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: How to detect page loading completion after leaving the

Post by chivracq » Wed Jun 22, 2016 12:41 am

serbeer wrote:Hello,
this is my first post as I've always been able to find my questions already answered on this forum, but not this time...

I am running

Code: Select all

FF v47 with Imacros plugin v8.9.7 on Windows 7 64-bit system. 
I am trying to speed up filling out 3 to 18 forms, with each open in its own Firefox tab by .iim script.

I currently use .iim script to
a)open with TAB OPEN, then T=1, then URL GOTO=http:\\... commands
b)waiting for page to finish loading based on standard SET !TIMEOUT_PAGE 60
c)fillout the form with number of TAGs
d)submit without waiting for results using

Code: Select all

SET !TIMEOUT_PAGE 1
SET !ERRORIGNORE YES
TAG POS=1 TYPE=BUTTON FORM=NAME:NoFormName ATTR=ID:search-button 
Then I repeat for tabs #2 and so on. That works perfectly fine, but is not very fast as it leads to 3-18 waits for each page load, one-by-one.

I realized that I could speed up this process considerably, by factor of 1.5 in my experiments, if I only pre-load each page by opening them one by one first to let them load in parallel breaking out before they finish loading, then fill them out one by one in the second "rotation" of each tab. Something like

Code: Select all

SET !TIMEOUT_PAGE 1
SET !ERRORIGNORE YES
a)open with TAB OPEN, then T=1, then URL GOTO=http:\\URL1.com
b)open with TAB OPEN, then T=2, then URL GOTO=http:\\URL2.com
' ...Then I repeat for each of 3-18 tabs, pre-fetching, then go back to fill out the forms
c)T=1
   SET !ERRORIGNORE NO
   SET !TIMEOUT_PAGE 60
   'fill out form in tab 1 here
d) submit without waiting for results with 
    SET !TIMEOUT_PAGE 1
    SET !ERRORIGNORE YES
   TAG POS=1 TYPE=BUTTON FORM=NAME:NoFormName ATTR=ID:search-button
e) T=2
   SET !ERRORIGNORE NO
   SET !TIMEOUT_PAGE 60
   'fill out form in tab 2 here
f) submit without waiting for results with 
    SET !TIMEOUT_PAGE 1
    SET !ERRORIGNORE YES
   TAG POS=1 TYPE=BUTTON FORM=NAME:NoFormName2 ATTR=ID:search-button-2
' ...Then I repeat for each of 3-18 tabs, filling out the forms on pages already hopefully loaded
As I said, that indeed results in much time savings, however, I now have problems with the fact that sometimes I came back to tab #1 (after opening all other tabs and initiating load in them) before the webpage in tab #1 had a chance to load fully. In such cases, my script frequently errors out trying to fill out form elements.

I tried introducing a few seconds delay after coming back to tab #1 with WAIT statement, but this is not efficient as sometimes I lose time on wait when the page is already loaded, and sometimes the delay proves insufficient. I tried to use
SET !TIMEOUT_STEP 10
TAG ... EXTRACT=TXT
but cannot find good element on the page to wait for, it is not obvious what loads in which order, and the pages are very complicated taking tens of seconds sometimes to load everything. Neither do I know of a way to find out what could be a reliable flag in native page full of CSS--nothing I found shows order of elements loading for a webpage. Perhaps you know of a tool to see loading order? Firebug only shows network communications timeline, not loading timeline.

On the other hand, Imacros has no problem detecting page loading completion correctly if I simply wait for URL GOTO=http:\\... step completion, as long as I am not breaking out using
SET !TIMEOUT_PAGE 1
SET !ERRORIGNORE YES
to pre-fetch other pages.

So, perhaps someone clever on this forum knows a way to make it detect/wait for page loading completion after coming back to the tab I broke out of before it had a chance to load fully? I am not sure what mechanism it normally uses to detect page load completion, but is there a way to "re-activate" it after coming back?

Or use some other trick/method of page loading completion detection after coming back to the tab?


Simply re-setting
SET !ERRORIGNORE NO
SET !TIMEOUT_PAGE 60
after re-entering the tab with load in progress has no effect, does not wait for page load completion.

I am already considering really tricky approaches, eg using Greasemonkey to inject some "flag" element into page upon loading completion which then I would be able to detect with Imacros TAG, eg:

Code: Select all

//Greasemonkey script
$(document).ready(function()
        {
            //add some flag element to the page here, perhaps in a DIV, like "Finished" link
        }
    );
but that approach is really complicated and will degrade performance since Greasemonkey will be running in each page. Is there something simpler in existence?

Hope someone here may have an advice for me. Thank you very much!
Compliments on the way you presented your Case, I only miss the URL('s) to be able to have a look myself and give you a more "educated" Advice, but opening and looping first through the 18 Tabs with a short '!TIMEOUT_PAGE' then going back to TAB_1 to process the Filling part would indeed also be my Approach.

Once back on TAB_1, you want to find a Mechanism to make sure the Page has finished loading. Forget about a hard-coded 'WAIT' for the Reasons that you mentioned, or you need to use a "Conditional WAIT" (Keywords to use to search my Posts) based on tagging one Element that you know will appear as the latest one on the Page but you mention yourself that you never know really in which Order they will load (and I don't know of a Tool to analyze the Load Order on a Page...), in that case, instead of tagging 1 Element, you could tag 3 or 5 Elements, each with their own Conditional WAIT.
But instead of using a Conditional WAIT, the Technique you are trying to use with a longer '!TIMEOUT_STEP' is correct as well and easier to implement, and again, instead of tagging 1 Element, tag 3 or 5, eventually with Relative Positioning, that will give you an extra 30 or 50 sec with even an extra long (60 sec) Conditional WAIT based on the 5th Element, then you can try to tag it again and if still not found you can either fire a "Conditional REFRESH" of the Page (using 'EVAL()' and 'URL GOTO'), hoping that this time the Page will load correctly with maybe again a Check on some Element(s) on the Page and/or you could trigger a "Conditional PROMPT" on that Page that will call your Attention that stg is going wrong on that Tab, followed by a few Seconds Conditional WAIT to allow you to manually pause the Macro while you manually reload the Page and you resume your Macro once the naughty Tab/Page has finally loaded correctly.
And if the Page/Tab had already correctly fully loaded, tagging the 5 Elements and deciding if a few Conditional WAIT's need to be computed would only take a fraction of a Second.
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE').
- I don't even read the Qt if that (required) Info is not mentioned...!
- Script & URL help a lot for more "educated" Help...
serbeer
Posts: 44
Joined: Fri Sep 11, 2015 5:36 am

Re: How to detect page loading completion after leaving the

Post by serbeer » Wed Jun 22, 2016 5:02 pm

Thank you very much for your response chivracq. And let me take this opportunity to thank you for your contribution to the forum--more than half of all IMacros-related printouts on my desk are printouts of your posts, most of them related to conditional processing in .iim (brilliant!) and keystrokes generation.

You propose VERY thorough error-handling strategy I must say, with some ideas I have not even considered before, like conditional prompting user about error with wait to let the macro be paused to resolve the error on web-site manually--this could indeed be huge for someone running complex multi-step script that sometimes errors at the end though probably an overkill for my application that normally completes in less than 5 minutes total, at least if I pre-fetch the sites.

Another idea you mentioned though, "fire a "Conditional REFRESH" of the Page (using 'EVAL()' and 'URL GOTO'), hoping that this time the Page will load correctly" is something that really interests me, even outside of scope of original question of this post. The thing is, it is not rare for one of these 18 sites to time out trying to load the page, usually after form submission or even in further post-processing step (perhaps on the 3rd or even 4th rotation across the tabs while revising results), which I do not handle at all at this time, the script stops and the remaining sites are simply not revised to completion. I did not want to complicate script too much, it has a few thousands of lines as is already, so did not replace waiting for all page loads with conditional waits using TAG elements with EVALs. But if I could somehow intercept timeout on page loading error and fire conditional refresh you mentioned in such case, that would have large impact on quality of processing. iim scripts do not have any exception handling though as far as I know, so timing out on page load simply terminates the script.

Still, I am hoping that I could introduce TAG EXTRACT-based conditional waits for page load followed by this Conditional Refresh that you mention, in places known to be the most troubling time-out-wise, as complexity not worth going into to simply bypass a problem could very well be worth going into if the problem can be fixed on the fly with conditional refresh. I searched through your posts but could not find any previous mention of Conditional Refresh. Perhaps you have an example I could use?

Finally, tagging multiple elements could indeed work for my OP problem, even though I am not 100% sure if you intend it to cover more ground in hope of finding a reliable load completion indicator element or buying time while the page would complete loading. The thing is, the page appears to be visually loaded at the time my scripts errors out, so the problem appears to be related to some internal elements not being ready. You are right about me failing to provide URL/example illustrating the issue, the timing issues are hard to illustrate consistently, so here is one short example with leaving a single tab and coming back to it immediately on a site that takes only several seconds to load but still may demonstrate the problem:

Code: Select all

VERSION BUILD=8970419
SET !REPLAYSPEED FAST
SET !EXTRACT_TEST_POPUP NO
'======================================Pre-open all sites
SET !TIMEOUT_PAGE 1
SET !ERRORIGNORE YES

TAB OPEN
TAB T=1
TAB CLOSEALLOTHERS
CLEAR
URL GOTO=http://www.expedia.com/Flights
'switch to next tab
TAB OPEN
TAB T=2
URL GOTO=http://www.google.com
SET !ERRORIGNORE NO
SET !TIMEOUT_PAGE 60
'======================================Come back to first tab and fill it out
TAB T=1
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:NoFormName ATTR=ID:*-origin CONTENT=ORD
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:NoFormName ATTR=ID:*-destination CONTENT=CUN
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:NoFormName ATTR=ID:*-departing CONTENT=07/23/2016
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:NoFormName ATTR=ID:*-returning CONTENT=07/30/2016
TAG POS=1 TYPE=SELECT FORM=NAME:NoFormName ATTR=ID:*-adults CONTENT=$2
TAG POS=1 TYPE=A ATTR=ID:advanced-options
TAG POS=1 TYPE=INPUT:CHECKBOX FORM=NAME:NoFormName ATTR=ID:advanced-*-nonstop CONTENT=YES
' Do not wait for results of Search submission
SET !TIMEOUT_PAGE 1
SET !ERRORIGNORE YES
TAG POS=1 TYPE=BUTTON FORM=NAME:NoFormName ATTR=ID:search-button
Running it will frequently result in “Please correct the errors below” red message which is what my OP was about and is the most bothersome since it means all TAGs went through without error but the form was not ready at submission time, and sometimes there is “RuntimeError: element INPUT specified by ID:advanced-*-nonstop was not found, line 27 (Error code: -921)”--that I could conditionally check before tagging, and sometimes the search starts successfully.

Thank you.
chivracq
Posts: 8871
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: How to detect page loading completion after leaving the

Post by chivracq » Thu Jun 23, 2016 6:18 pm

serbeer wrote:Thank you very much for your response chivracq. And let me take this opportunity to thank you for your contribution to the forum--more than half of all IMacros-related printouts on my desk are printouts of your posts, most of them related to conditional processing in .iim (brilliant!) and keystrokes generation.
Ah-ah-ah...!, Thanks for your Appreciation... 8)
You just need to be a bit Creative and you can do great things with iMacros, ah-ah...! And I do all my Scripts in pure '.iim', I don't use '.js' Scripts.
serbeer wrote:You propose VERY thorough error-handling strategy I must say, with some ideas I have not even considered before, like conditional prompting user about error with wait to let the macro be paused to resolve the error on web-site manually--this could indeed be huge for someone running complex multi-step script that sometimes errors at the end though probably an overkill for my application that normally completes in less than 5 minutes total, at least if I pre-fetch the sites.
Yep, nearly all my own Scripts include a heavy Level of Checks to ensure Reliability during the Run but with some Reporting as well by combining 'MacroError()' (that I "misuse" to display a mini-Report at the End of many of my Macros in the Side Panel) + (Conditional) 'PROMPT' + Logging and only a few of my Scripts go for Absolute Speed, even if all my Scripts are usually optimized for the best Performance already and some even tune themselves automatically depending on the Connection Speed / Nb of Elements to process on a Page / Nb of Loops / Time of the Day/Night and are able to adjust/toggle automatically '!ERRORIGNORE', '!TIMEOUT_PAGE', '!TIMEOUT_STEP', 'FILTER', 'WAIT'...

For your Case, I had even thought at the time of my first Reply to add a (Conditional) 3rd Round based on some mini-Report from Round_2 to handle only the "difficult" Tabs but that was based on thinking that each Tab had its 'CONTENT' Data stored and fetched from a '.CSV' DataSource, then it was easy to link TAB_n with '!DATASOURCE_LINE' and I'm a bit surprised to see that your Data is actually hard-coded in your Sample Script. (But maybe you hard-coded it for me to be able to run it...?)
serbeer wrote:Another idea you mentioned though, "fire a "Conditional REFRESH" of the Page (using 'EVAL()' and 'URL GOTO'), hoping that this time the Page will load correctly" is something that really interests me, even outside of scope of original question of this post. The thing is, it is not rare for one of these 18 sites to time out trying to load the page, usually after form submission or even in further post-processing step (perhaps on the 3rd or even 4th rotation across the tabs while revising results), which I do not handle at all at this time, the script stops and the remaining sites are simply not revised to completion. I did not want to complicate script too much, it has a few thousands of lines as is already, so did not replace waiting for all page loads with conditional waits using TAG elements with EVALs. But if I could somehow intercept timeout on page loading error and fire conditional refresh you mentioned in such case, that would have large impact on quality of processing. iim scripts do not have any exception handling though as far as I know, so timing out on page load simply terminates the script.

Still, I am hoping that I could introduce TAG EXTRACT-based conditional waits for page load followed by this Conditional Refresh that you mention, in places known to be the most troubling time-out-wise, as complexity not worth going into to simply bypass a problem could very well be worth going into if the problem can be fixed on the fly with conditional refresh. I searched through your posts but could not find any previous mention of Conditional Refresh. Perhaps you have an example I could use?
What I call a "Conditional REFRESH" (and I thought I had already posted some Example) is actually a "Conditional 'URL GOTO'" and the Principle is the same like for all "my" Conditional 'WAIT' / 'SAVEAS' / 'TAG', except 'PROMPT' which uses a different Mechanism because of a Bug in iMacros for FF: Using 'EVAL()' and some Condition on most of the time '!EXTRACT', compute a 1/0/20 or a String/Empty_String to reuse for the Command, so a Conditional REFRESH for your Sample Script would look like this:

Code: Select all

VERSION BUILD=8970419
SET !EXTRACT_TEST_POPUP NO
SET !TIMEOUT_PAGE 1
SET !ERRORIGNORE YES

SET URL_Site1 http://www.expedia.com/Flights

'URL GOTO=http://www.expedia.com/Flights
URL GOTO={{URL_Site1}}

SET !TIMEOUT_STEP 0
TAG POS=1 TYPE=BUTTON FORM=NAME:NoFormName ATTR=ID:search-button EXTRACT=TXT
SET URL_Refresh EVAL("var s='{{!EXTRACT}}'; var u='{{URL_Site1}}'; var x; if(s=='#EANF#'){x=u;} else {x='';}; x;")
'PROMPT _{{URL_Refresh}}_
WAIT SECONDS=2
PROMPT "I guess the 'Search' Button was probably not found, Site will refresh in 3 sec...!"
URL GOTO={{URL_Refresh}}
(Tested on iMacros for FF v8.8.2, Pale Moon v26.1.1, Win10-x64.)

I ran my mini-Script about 10 times, only once the Site had the time to load in less than 1 sec for the 'Search' Button to be found (and the Page didn't refresh then). Increase '!TIMEOUT_PAGE' and/or '!TIMEOUT_STEP' for the 'Search' Button to be found...
serbeer wrote:Finally, tagging multiple elements could indeed work for my OP problem, even though I am not 100% sure if you intend it to cover more ground in hope of finding a reliable load completion indicator element or buying time while the page would complete loading. The thing is, the page appears to be visually loaded at the time my scripts errors out, so the problem appears to be related to some internal elements not being ready. You are right about me failing to provide URL/example illustrating the issue, the timing issues are hard to illustrate consistently, so here is one short example with leaving a single tab and coming back to it immediately on a site that takes only several seconds to load but still may demonstrate the problem:

Code: Select all

VERSION BUILD=8970419
SET !REPLAYSPEED FAST
SET !EXTRACT_TEST_POPUP NO
'======================================Pre-open all sites
SET !TIMEOUT_PAGE 1
SET !ERRORIGNORE YES

TAB OPEN
TAB T=1
TAB CLOSEALLOTHERS
CLEAR
URL GOTO=http://www.expedia.com/Flights
'switch to next tab
TAB OPEN
TAB T=2
URL GOTO=http://www.google.com
SET !ERRORIGNORE NO
SET !TIMEOUT_PAGE 60
'======================================Come back to first tab and fill it out
TAB T=1
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:NoFormName ATTR=ID:*-origin CONTENT=ORD
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:NoFormName ATTR=ID:*-destination CONTENT=CUN
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:NoFormName ATTR=ID:*-departing CONTENT=07/23/2016
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:NoFormName ATTR=ID:*-returning CONTENT=07/30/2016
TAG POS=1 TYPE=SELECT FORM=NAME:NoFormName ATTR=ID:*-adults CONTENT=$2
TAG POS=1 TYPE=A ATTR=ID:advanced-options
TAG POS=1 TYPE=INPUT:CHECKBOX FORM=NAME:NoFormName ATTR=ID:advanced-*-nonstop CONTENT=YES
' Do not wait for results of Search submission
SET !TIMEOUT_PAGE 1
SET !ERRORIGNORE YES
TAG POS=1 TYPE=BUTTON FORM=NAME:NoFormName ATTR=ID:search-button
Running it will frequently result in “Please correct the errors below” red message which is what my OP was about and is the most bothersome since it means all TAGs went through without error but the form was not ready at submission time, and sometimes there is “RuntimeError: element INPUT specified by ID:advanced-*-nonstop was not found, line 27 (Error code: -921)”--that I could conditionally check before tagging, and sometimes the search starts successfully.

Thank you.
"... even though I am not 100% sure if you intend it to cover more ground in hope of finding a reliable load completion indicator element or buying time while the page would complete loading."
=> Yep, both...!

The INPUT Field triggering the RuntimeError is then definitely one of the Elements to include in the Checks.

But hum, I thought of a Tool that you can use/misuse to discover the Load Order of Elements/Components on a Web-Page: AdBlock (ABP or ABE or uBlock), the Blockable Elements/Components on a Page are listed in the Order that they get loaded/discovered by ABE... :idea:
Last edited by chivracq on Fri Jun 24, 2016 2:16 am, edited 2 times in total.
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE').
- I don't even read the Qt if that (required) Info is not mentioned...!
- Script & URL help a lot for more "educated" Help...
serbeer
Posts: 44
Joined: Fri Sep 11, 2015 5:36 am

Re: How to detect page loading completion after leaving the

Post by serbeer » Thu Jun 23, 2016 11:54 pm

chivracq wrote:And I do all my Scripts in pure '.iim', I don't use '.js' Scripts.
chivracq wrote: Yep, nearly all my own Scripts include a heavy Level of Checks to ensure Reliability during the Run but with some Reporting as well by combining 'MacroError()' (that I "misuse" to display a mini-Report at the End of many of my Macros in the Side Panel) + (Conditional) 'PROMPT' + Logging and only a few of my Scripts go for Absolute Speed, even if all my Scripts are usually optimized for the best Performance already and some even tune themselves automatically depending on the Connection Speed / Nb of Elements to process on a Page / Nb of Loops / Time of the Day/Night and are able to adjust/toggle automatically '!ERRORIGNORE', '!TIMEOUT_PAGE', '!TIMEOUT_STEP', 'FILTER', 'WAIT'...
chivracq wrote:...thinking that each Tab had its 'CONTENT' Data stored and fetched from a '.CSV' DataSource, then it was easy to link TAB_n with '!DATASOURCE_LINE' and I'm a bit surprised to see that your Data is actually hard-coded in your Sample Script. (But maybe you hard-coded it for me to be able to run it...?)
Pretty cool. Yes, this was hardcoded for smallest possible sample size. For my real app, I generate huge .iim script from Javascript running off form loaded as local html page, aka control panel, that user uses for data entry and to set control options for .iim generation. All inputs are vars picked from this control panel by .iim script. Javascript is mostly used to avoid too many variables in .iim that would make code hard to read, by only concatenating together the tabs for sites that user choses to process. I have log file written for every run detailing all info used, and take snapshot of control panel on top of it as well. I also capture any errors if any in the log and take snapshop of the screen and web page if the script errored out--so that I can debug it later. All from .iim, not Javascript--and save .iim script itself too. The heart is still .iim script, and it does have some of the same features your does, with timeouts set based on number of tabs opened, image filter if connection is slow, etc, but needs to be made more conditionally robust, you gave me a few new ideas.
chivracq wrote: ... so a Conditional REFRESH for your Sample Script would look like this:
VERSION BUILD=8970419
SET !EXTRACT_TEST_POPUP NO
SET !TIMEOUT_PAGE 1
SET !ERRORIGNORE YES

SET URL_Site1 http://www.expedia.com/Flights

'URL GOTO=http://www.expedia.com/Flights
URL GOTO={{URL_Site1}}

SET !TIMEOUT_STEP 0
TAG POS=1 TYPE=BUTTON FORM=NAME:NoFormName ATTR=ID:search-button EXTRACT=TXT
SET URL_Refresh EVAL("var s='{{!EXTRACT}}'; var u='{{URL_Site1}}'; var x; if(s=='#EANF#'){x=u;} else {x='';}; x;")
'PROMPT _{{URL_Refresh}}_
WAIT SECONDS=2
PROMPT "I guess the 'Search' Button was probably not found, Site will refresh in 3 sec...!"
URL GOTO={{URL_Refresh}}[/code](Tested on iMacros for FF v8.8.2, Pale Moon v26.1.1, Win10-x64.)

I ran my mini-Script about 10 times, only once the Site had the time to load in less than 1 sec for the 'Search' Button to be found (and the Page didn't refresh then). Increase '!TIMEOUT_PAGE' and/or '!TIMEOUT_STEP' for the 'Search' Button to be found...
Thank you for the example! It is indeed accomplishes refresh of original site, so is indeed applicable to OP problem resolution. I was secretly hoping that you found some way to execute actual REFRESH command conditionally because reloading URL can only work for the very first stage of processing, while if REFRESH command could be executed, it would be applicable to later stages, for example, if the form has been filled out and submitted but page did not finish loading, then REFRESH would it another attempt (at least if GET is used vs POST) while GOTO URL would only reload original empty form. Oh well :)
chivracq wrote: But hum, I thought of a Tool that you can use/misuse to discover the Load Order of Elements/Components on a Web-Page: AdBlock (ABP or ABE or uBlock), the Blockable Elements/Components on a Page are listed in the Order that they get loaded/discovered by ABE... :idea:
Interesting idea, thank you. I experimented with this using this very same URL. It appears that AdBlock Plus that I have installed on my FF only lists images, scripts, fonts, stylesheets, and XML requests as blockable elements. No actual HTML. Of course, images could be used through image match with paid version and IMacro browser, but that is not nearly as good as attributes of html elements since would require storing images and I prefer to stick to FF version of IMacros anyway. So if I am missing something and not using AdBlock Plus correctly, please let me know.

Anyway, thank you for your help with this, I will give it another try soon.
Post Reply