URL GOTO: sometimes nothing happens

Discussions and Tech Support specific to the iMacros Firefox add-on.
Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information: CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
Post Reply
sightunseen
Posts: 67
Joined: Fri Apr 19, 2013 4:54 am

URL GOTO: sometimes nothing happens

Post by sightunseen » Sun Jun 14, 2015 4:06 am

I'm using iMacros 8.9.2.1 in the latest Firefox Web Developer Edition. I have a script that I run in a loop and it starts like this:

WAIT SECONDS=5
TAB T=1
URL GOTO=http://website
TAG POS=1 TYPE=FONT ATTR=TXT:text* EXTRACT=TXT

The script then submits a form on that page which causes another page to load, then the script is done and it loops back around.

The problem is rarely it doesn't GOTO website. I will come back to the computer and iMacros will be stuck at the next step waiting for the TAG (I have timeout set to infinite). But the page isn't loaded or even in the process of loading and the url is unchanged (it's still the form result page). This happened once today and once yesterday. Today after about 50 repeats and yesterday after some larger number but likely less than 100.

I think this is a bug unless it actually goes to the page and that page for some reason redirects to the form result page without the form being filled out? Is there a way I can log detailed information so I can figure out whether this is a bug in iMacros or a problem with the website?
new movies
Posts: 44
Joined: Thu May 26, 2011 8:14 pm

Re: URL GOTO: sometimes nothing happens

Post by new movies » Sun Jun 14, 2015 7:30 am

Some bugs happen in old while some in new.
Try changing firefox version to an old one .
Or try imacros old addon with old browser
chivracq
Posts: 9419
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: URL GOTO: sometimes nothing happens

Post by chivracq » Sun Jun 14, 2015 2:36 pm

sightunseen wrote:I'm using iMacros 8.9.2.1 in the latest Firefox Web Developer Edition. I have a script that I run in a loop and it starts like this:

WAIT SECONDS=5
TAB T=1
URL GOTO=http://website
TAG POS=1 TYPE=FONT ATTR=TXT:text* EXTRACT=TXT

The script then submits a form on that page which causes another page to load, then the script is done and it loops back around.

The problem is rarely it doesn't GOTO website. I will come back to the computer and iMacros will be stuck at the next step waiting for the TAG (I have timeout set to infinite). But the page isn't loaded or even in the process of loading and the url is unchanged (it's still the form result page). This happened once today and once yesterday. Today after about 50 repeats and yesterday after some larger number but likely less than 100.

I think this is a bug unless it actually goes to the page and that page for some reason redirects to the form result page without the form being filled out? Is there a way I can log detailed information so I can figure out whether this is a bug in iMacros or a problem with the website?
FCIM...! :mrgreen: => iMacros for FF v8.9.2, FF38/39...?, OS...?
"Latest Version" is always vague as you always have 3 "latest" FF Versions... and it won't mean anything to sbd reading the Thread in a few weeks/months/years...

A few Remarks/Suggestions:
- Yep, I don't find it very clever to have '!TIMEOUT' set to Infinite, you know that if a Page has not loaded within 60 sec, there is already a Pb, because if anything happens like it happens to you sometimes, your Script hangs, and it doesn't matter if the Pb comes from iMacros or from the Web-Site, you know there is a Pb somewhere, you should make your Script more "robust", by checking the '!URLCURRENT' if it's the one you expect, and/or by tagging/extracting some Field you know should be present on the Page before processing any further, and you can log all that Info (with Timestamp) in some Custom Log File of yours.

- You can trigger a Conditional Sound if stg goes wrong, or a Conditional Promp/Alert, you could even have your Script send you an automatic Mail or SMS...

- You can schedule your Script to automatically start again (and kill the existing FF Instance eventually) every hour...

- On FF, there is somewhere a Global Log File (mentioned in several (older) Threads), but I've never been able to locate it... (I didn't do much Digging either...) On IE you can specify a specific Log File...

- You could enable the Profiler which logs quite some Data about your Macro running...

- And as mentioned in my first Item, you can do your own Custom Logging... :idea:
(I just had the case, yesterday/today with a Web-Site being down for about 24h, I was the one noticing and pinging the Developer/Admin of the Site and able to tell him from looking at my Log when the Pb started...)

- Maybe using Alertfox is an Option for you...
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE').
- I don't even read the Qt if that (required) Info is not mentioned...!
- Script & URL help a lot for more "educated" Help...
sightunseen
Posts: 67
Joined: Fri Apr 19, 2013 4:54 am

Re: URL GOTO: sometimes nothing happens

Post by sightunseen » Mon Jun 15, 2015 4:05 am

chivracq wrote:FCIM...! :mrgreen: => iMacros for FF v8.9.2, FF38/39...?, OS...?
"Latest Version" is always vague as you always have 3 "latest" FF Versions... and it won't mean anything to sbd reading the Thread in a few weeks/months/years...

A few Remarks/Suggestions:
- Yep, I don't find it very clever to have '!TIMEOUT' set to Infinite, you know that if a Page has not loaded within 60 sec, there is already a Pb, because if anything happens like it happens to you sometimes, your Script hangs, and it doesn't matter if the Pb comes from iMacros or from the Web-Site, you know there is a Pb somewhere, you should make your Script more "robust", by checking the '!URLCURRENT' if it's the one you expect, and/or by tagging/extracting some Field you know should be present on the Page before processing any further, and you can log all that Info (with Timestamp) in some Custom Log File of yours.
Hi chivracq nice to hear from you and thanks for all your suggestions. I'm sure I could work around this but if it's a bug I want to report it and get it fixed so that leaves logging to see what's happening. I'm using Firefox Web Developer 40.0a2 (2015-06-11) on Windows 7 x64. What is a Pb? There is a tag for the field I know is on the page and that's what the script hangs on, waiting for that tag which never arrives because (it appears) iMacros didn't go to the URL.
chivracq
Posts: 9419
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: URL GOTO: sometimes nothing happens

Post by chivracq » Mon Jun 15, 2015 7:27 am

Ah OK, FF40 already...!

Pb = Problem

OK for wanting to report the Pb if there is a Bug indeed..., but a Bug that only happens sometimes is always difficult to reproduce...

In your case shorten '!TIMEOUT(_PAGE)', and play with '!TIMEOUT_STEP' and '!ERRORIGNORE'...
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE').
- I don't even read the Qt if that (required) Info is not mentioned...!
- Script & URL help a lot for more "educated" Help...
sightunseen
Posts: 67
Joined: Fri Apr 19, 2013 4:54 am

Re: URL GOTO: sometimes nothing happens

Post by sightunseen » Thu Jun 25, 2015 4:26 am

chivracq wrote:OK for wanting to report the Pb if there is a Bug indeed..., but a Bug that only happens sometimes is always difficult to reproduce...

In your case shorten '!TIMEOUT(_PAGE)', and play with '!TIMEOUT_STEP' and '!ERRORIGNORE'...
Thanks. I don't want to shorten the timeout though, I want it to wait as long as possible. Also I don't want to ignore an error. It just happened again so I'm going to use a proxy to log all the requests my browser makes. I'll let everyone know what happens.
Cobra99
Posts: 64
Joined: Thu Dec 25, 2014 12:17 am

Re: URL GOTO: sometimes nothing happens

Post by Cobra99 » Sat Jun 27, 2015 1:09 am

Use CLEAR command
Try where to place it - as first line on each loop or before SUBMIT button...
VERSION BUILD=8820413 RECORDER=FX, WIN7 64-bit SK, Pale Moon 25.8.1(x86)
FCIM here http://forum.imacros.net/viewtopic.php?f=20&t=3331
sightunseen
Posts: 67
Joined: Fri Apr 19, 2013 4:54 am

Re: URL GOTO: sometimes nothing happens

Post by sightunseen » Sat Jun 27, 2015 1:53 am

Cobra99 wrote:Use CLEAR command
Try where to place it - as first line on each loop or before SUBMIT button...
No thanks I don't want to wipe my cache and cookies.
sightunseen
Posts: 67
Joined: Fri Apr 19, 2013 4:54 am

Re: URL GOTO: sometimes nothing happens

Post by sightunseen » Sun Jul 05, 2015 3:17 am

The proxy shows that Firefox has requested the page but for some reason the page is not updated in the tab and iMacros ends up waiting for the tag. I was able to confirm this by modifying my iim script and putting this line right before URL GOTO:

SET !CLIPBOARD "Going to URL {{!NOW:mm-dd__hh.nn.ss}}"

When the script got stuck waiting for the tag after the URL GOTO I was able to determine the timestamp when the URL was requested and then confirm that timestamp in the proxy that Firefox had requested that URL. The URL had the same content as any other time but the tab isn't updated. I don't know why that is. Does anyone else have this problem?
chivracq
Posts: 9419
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: URL GOTO: sometimes nothing happens

Post by chivracq » Mon Jul 06, 2015 5:14 am

Yep again..., make your Scripts more robust... Internet has never been 100% reliable, now 2000% more than 10 years ago, but still not 100% reliable...
I don't want to disclose all my Techniques but my iMacros Scripts palliate all possible Internet/Connection/Server Errors... (well, if I have a Connection of course...!)

I think you are clever enough to find your own Workarounds, but in one of my Scripts (run from Win-Scheduler every n hours), next to checking if the Page (well all Pages...) has/have loaded correctly, if anything goes wrong, I have a MacroError (= 'EVAL()') aborting the Macro after it has written to some Temp File a "Go" for the same Macro to eventually run again (and check for "Go") 10 min later... But I could as well have chosen to rerun the same Macro Conditionally 1-2-5-10 times directly from the .iim Macro itself...
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE').
- I don't even read the Qt if that (required) Info is not mentioned...!
- Script & URL help a lot for more "educated" Help...
sightunseen
Posts: 67
Joined: Fri Apr 19, 2013 4:54 am

Re: URL GOTO: sometimes nothing happens

Post by sightunseen » Mon Jul 06, 2015 5:41 am

chivracq wrote:Yep again..., make your Scripts more robust... Internet has never been 100% reliable, now 2000% more than 10 years ago, but still not 100% reliable...
I don't want to disclose all my Techniques but my iMacros Scripts palliate all possible Internet/Connection/Server Errors... (well, if I have a Connection of course...!)

I think you are clever enough to find your own Workarounds, but in one of my Scripts (run from Win-Scheduler every n hours), next to checking if the Page (well all Pages...) has/have loaded correctly, if anything goes wrong, I have a MacroError (= 'EVAL()') aborting the Macro after it has written to some Temp File a "Go" for the same Macro to eventually run again (and check for "Go") 10 min later... But I could as well have chosen to rerun the same Macro Conditionally 1-2-5-10 times directly from the .iim Macro itself...
Thanks for the information. I think it's a bug in iMacros or a bug in Firefox, I don't know which. My scripts always use to run without incident when I would loop them.
chivracq
Posts: 9419
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: URL GOTO: sometimes nothing happens

Post by chivracq » Mon Jul 06, 2015 6:46 am

sightunseen wrote:
chivracq wrote:Yep again..., make your Scripts more robust... Internet has never been 100% reliable, now 2000% more than 10 years ago, but still not 100% reliable...
I don't want to disclose all my Techniques but my iMacros Scripts palliate all possible Internet/Connection/Server Errors... (well, if I have a Connection of course...!)

I think you are clever enough to find your own Workarounds, but in one of my Scripts (run from Win-Scheduler every n hours), next to checking if the Page (well all Pages...) has/have loaded correctly, if anything goes wrong, I have a MacroError (= 'EVAL()') aborting the Macro after it has written to some Temp File a "Go" for the same Macro to eventually run again (and check for "Go") 10 min later... But I could as well have chosen to rerun the same Macro Conditionally 1-2-5-10 times directly from the .iim Macro itself...
Thanks for the information. I think it's a bug in iMacros or a bug in Firefox, I don't know which. My scripts always use to run without incident when I would loop them.
Yep, then post a Script to reproduce your Pb..., I re-read the whole Thread but you don't post the URL of your "specific" Site... I use iMacros with many Sites and I've never experienced a Pb (for about 10 years now, and 'URL GOTO' works very fine...!) that I would relate to FF or iMacros, it's always the Web-Server which is faulty if there is any Pb...
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE').
- I don't even read the Qt if that (required) Info is not mentioned...!
- Script & URL help a lot for more "educated" Help...
sightunseen
Posts: 67
Joined: Fri Apr 19, 2013 4:54 am

Re: URL GOTO: sometimes nothing happens

Post by sightunseen » Sun Aug 02, 2015 2:25 am

chivracq wrote:Yep, then post a Script to reproduce your Pb..., I re-read the whole Thread but you don't post the URL of your "specific" Site... I use iMacros with many Sites and I've never experienced a Pb (for about 10 years now, and 'URL GOTO' works very fine...!) that I would relate to FF or iMacros, it's always the Web-Server which is faulty if there is any Pb...
Unfortunately I cannot offer a link to this particular page to reproduce. This happened again today. I have continued to proxy my requests so that I could get a record if it happened again. This time at exactly the time iMacros goes to the URL the proxy doesn't show it. Instead for that exact time it shows a POST request for the form submission which causes a redirect. I think what happens is there is some sort of collision where if iMacros does a URL GOTO at the exact time a POST request is taking place (or maybe in the process of receiving the reply to the post and then getting the redirect -- more likely since I have WAITs in my script) that it will override the URL GOTO. Since nobody else has posted I will keep investigating. Next I am going to use something to record the screen so I can see what it looks like as it's happening.
sightunseen
Posts: 67
Joined: Fri Apr 19, 2013 4:54 am

Re: URL GOTO: sometimes nothing happens

Post by sightunseen » Tue Aug 04, 2015 12:51 am

sightunseen wrote:Since nobody else has posted I will keep investigating. Next I am going to use something to record the screen so I can see what it looks like as it's happening.
I captured it today in video. Unfortunately it contains proprietary information and I cannot share it. The video shows that at the exact moment iMacros executes the URL GOTO the tab switches to 'Connecting...' and that circle thing is spinning from the previous submission. The URL in the address bar then changes, not to what is in the URL GOTO but the landing page for completion of the form submission, and at that point iMacros continues to the next statement which is waiting for a tag. However as I mentioned earlier that waiting fails because it's not at the actual page it should be.

Based on the video it looks like after iMacros submits the form there is some javascript that can take a random amount of time (a few seconds) to complete before the form is actually posted. I need to account for this in my code so I will add a longer delay as a workaround, so that when I loop the script enough time has passed that the form has posted.

I purposely played with the wait times to make the issue a little bit reproducible and I tried to make some other workarounds in iMacros like setting document.location via URL GOTO which has the same problem as a regular URL GOTO. I even tried a setTimeout and waiting every 5 seconds to do a document.location. That works but not as expected because if I'm already on the form submission page then the tag is found and the script continues (but the previous form is still in the process of being submitted).

I hope this is enough information for iMacros developers to address this. If the document location can't be changed because it's in the process of changing (due to a form submission or something) there should be some way to wait until the location can be changed and then iMacros can change it.
Post Reply