Trying to redo loop if EANF
Forum rules
iMacros EOL - Attention!
The renewal maintenance has officially ended for Progress iMacros effective November 20, 2023 and all versions of iMacros are now considered EOL (End-of-Life). The iMacros products will no longer be supported by Progress (aside from customer license issues), and these forums will also no longer be moderated from the Progress side.
Thank you again for your business and support.
Sincerely,
The Progress Team
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information: CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
iMacros EOL - Attention!
The renewal maintenance has officially ended for Progress iMacros effective November 20, 2023 and all versions of iMacros are now considered EOL (End-of-Life). The iMacros products will no longer be supported by Progress (aside from customer license issues), and these forums will also no longer be moderated from the Progress side.
Thank you again for your business and support.
Sincerely,
The Progress Team
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information: CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
-
- Posts: 5
- Joined: Mon Aug 03, 2020 10:36 pm
Trying to redo loop if EANF
Hi guys, first post, trying to self teach iim and its going pretty well but i need to redo some of my web scraping and im a bit stumped. Heres the raw code, I use it to enter 2 variables and then press go and i want to scrape the output The issue is My internet is rotating Ip every so often and if it times up just right when it rotates then the output is EANF, for some reason if i get too many of those then iim just locks up and it doesn't work anymore, I like to set this up to run all night and set it to run slow so it never messes up. I think what i need to do though is something along the lines of
If Extract = #EANF# then redo this loop using same variables
SAVEAS TYPE=EXTRACT FOLDER=C:\Users\hirea\Downloads FILE=Output.csv
This is the line that extracts the output and occasionally cant find the expected Anchor, I want some sort of test at this point in the loop that says if #EANF# then go back up to "TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:"first Variable" CONTENT={{!COL1}}"
Heres my config
Windows 10 Pro Version 1909 English
imacros Browser V12.6.505.4525 have license but was using Internet explorer plugin and not sure how to migrate liscense
Demos seem to work
Not sure how to test VBS scripting interface
get the same issue in IE explorer and imacros browser The issue is my cycline IP kills the internet for a few seconds and inevitably my iim is trying to scrap data at that exact moment
VERSION BUILD=12.5.2018.1105
'
URL GOTO="the website im scraping"
SET !DATASOURCE C:\Users\hirea\Desktop\"the file with the 2 inputs"
SET !LOOP 1
'Increase the current position in the file with each loop
SET !DATASOURCE_LINE {{!LOOP}}
SET !WAITPAGECOMPLETE no
TAB T=1
SET !ERRORIGNORE YES
TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:"first Variable" CONTENT={{!COL1}}
TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:"2nd Variable" CONTENT={{!COL2}}
WAIT SECONDS=2
TAG POS=1 TYPE=BUTTON:SUBMIT ATTR=TXT:"Give me the output"
ADD !EXTRACT {{!COL1}}
ADD !EXTRACT {{!COL2}}
WAIT SECONDS=2
TAG POS=1 TYPE=DIV ATTR=CLASS:"output" EXTRACT=TXT
WAIT SECONDS=2
SAVEAS TYPE=EXTRACT FOLDER=C:\Users\hirea\Downloads FILE=Output.csv
URL GOTO="the website im scraping"
WAIT SECONDS=6
If Extract = #EANF# then redo this loop using same variables
SAVEAS TYPE=EXTRACT FOLDER=C:\Users\hirea\Downloads FILE=Output.csv
This is the line that extracts the output and occasionally cant find the expected Anchor, I want some sort of test at this point in the loop that says if #EANF# then go back up to "TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:"first Variable" CONTENT={{!COL1}}"
Heres my config
Windows 10 Pro Version 1909 English
imacros Browser V12.6.505.4525 have license but was using Internet explorer plugin and not sure how to migrate liscense
Demos seem to work
Not sure how to test VBS scripting interface
get the same issue in IE explorer and imacros browser The issue is my cycline IP kills the internet for a few seconds and inevitably my iim is trying to scrap data at that exact moment
VERSION BUILD=12.5.2018.1105
'
URL GOTO="the website im scraping"
SET !DATASOURCE C:\Users\hirea\Desktop\"the file with the 2 inputs"
SET !LOOP 1
'Increase the current position in the file with each loop
SET !DATASOURCE_LINE {{!LOOP}}
SET !WAITPAGECOMPLETE no
TAB T=1
SET !ERRORIGNORE YES
TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:"first Variable" CONTENT={{!COL1}}
TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:"2nd Variable" CONTENT={{!COL2}}
WAIT SECONDS=2
TAG POS=1 TYPE=BUTTON:SUBMIT ATTR=TXT:"Give me the output"
ADD !EXTRACT {{!COL1}}
ADD !EXTRACT {{!COL2}}
WAIT SECONDS=2
TAG POS=1 TYPE=DIV ATTR=CLASS:"output" EXTRACT=TXT
WAIT SECONDS=2
SAVEAS TYPE=EXTRACT FOLDER=C:\Users\hirea\Downloads FILE=Output.csv
URL GOTO="the website im scraping"
WAIT SECONDS=6
Last edited by NotguyinVideo on Tue Aug 04, 2020 12:09 pm, edited 1 time in total.
Re: Trying to redo loop if EANF
NotguyinVideo wrote: ↑Mon Aug 03, 2020 10:45 pmHi guys, first post, trying to self teach iim and its going pretty well but i need to redo some of my web scraping and im a bit stumped. Heres the raw code, I use it to enter 2 variables and then press go and i want to scrape the output The issue is My internet is rotating Ip every so often and if it times up just right when it rotates then the output is EANF, for some reason if i get too many of those then iim just locks up and it doesn't work anymore, I like to set this up to run all night and set it to run slow so it never messes up. I think what i need to do though is something along the lines of
This is the line that extracts the output and occasionally cant find the expected Anchor, I want some sort of test at this point in the loop that says if #EANF# then go back up to "TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:"first Variable" CONTENT={{!COL1}}"Code: Select all
If Extract = #EANF# then redo this loop using same variables SAVEAS TYPE=EXTRACT FOLDER=C:\Users\hirea\Downloads FILE=Output.csv
Code: Select all
VERSION BUILD=12.5.2018.1105 URL GOTO="the website im scraping" SET !DATASOURCE C:\Users\hirea\Desktop\"the file with the 2 inputs" ' SET !LOOP 1 'Increase the current position in the file with each loop SET !DATASOURCE_LINE {{!LOOP}} ' SET !WAITPAGECOMPLETE no ' TAB T=1 ' SET !ERRORIGNORE YES ' TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:"first Variable" CONTENT={{!COL1}} TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:"2nd Variable" CONTENT={{!COL2}} WAIT SECONDS=2 TAG POS=1 TYPE=BUTTON:SUBMIT ATTR=TXT:"Give me the output" ADD !EXTRACT {{!COL1}} ADD !EXTRACT {{!COL2}} WAIT SECONDS=2 TAG POS=1 TYPE=DIV ATTR=CLASS:"output" EXTRACT=TXT ' WAIT SECONDS=2 ' SAVEAS TYPE=EXTRACT FOLDER=C:\Users\hirea\Downloads FILE=Output.csv ' URL GOTO="the website im scraping" WAIT SECONDS=6
FCIM...! => for me to read, read my Sig..., apart from checking your Post for Spam before approving it...
And hum..., most Scripts posted are usually too compact, but yours is on the contrary too "one-Line-by-one-Line" with too many extra=unnecessary (empty) Lines..., => can you compact it a little bit, to make it readable "by Block"...?
("Readability" is one of the most important Factors on a Tech Forum, for (Advanced) Users trying to hep you (from your OP), and if the Thread gets a Solution, for other Users searching the Forum with a similar Qt/Pb/Scenario... )
>>>
And I have very bad Int. Co., not sure when I'll be able to check the Forum (and your Thread) again...
(Int. Co. now from a Friend tethering for me...)
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
-
- Posts: 5
- Joined: Mon Aug 03, 2020 10:36 pm
Re: Trying to redo loop if EANF
Sorry, went back in and provided more info and tried to clean up my coding
Re: Trying to redo loop if EANF
NotguyinVideo wrote: ↑Tue Aug 04, 2020 12:10 pmSorry, went back in and provided more info and tried to clean up my coding
OK, seen that, perfect, and Thanks...
And "sorry", but I don't even read en don't try to do any "Thinking" when FCI is not (clearly) mentioned... It's usually a "Waste of Time" for "everybody" otherwise, as I then need to go into several Layers of if-if-if, and you won't understand half of my Post(s), ah-ah...!
(... Like hum..., this is I'm "afraid" probably going to happen in this Thread very similar to yours actually, but where the User is not really "able" to mention their FCI...)
>>>
NotguyinVideo wrote: ↑Mon Aug 03, 2020 10:45 pmHeres my confighave license but was using Internet explorer plugin and not sure how to migrate liscenseCode: Select all
Windows 10 Pro Version 1909 English imacros Browser V12.6.505.4525
Demos seem to work
Not sure how to test VBS scripting interface
get the same issue in IE explorer and imacros browser The issue is my cycline IP kills the internet for a few seconds and inevitably my iim is trying to scrap data at that exact moment
Alright good, I don't requote your Script that you've compacted (nearly "too much" now, but OK, never mind...) to keep the Thread "readable"...
So, OK, you are then on iMB v12.6 'Trial' + Win10_Pro_#1909_Eng if I understand correctly...
And hum..., you'll have to "solve" the Licensing Issue, as if you are now on iMB v12.6 'Trial', the Trial Popups will soon start "nagging" you, and they will halt your Script while running "all night long" until you dismiss the Popup(s), defeating that Purpose...
You have a full Section in the FAQ about that kind of Licensing "Scenarios" and migrating/transferring a License from one PC/Installation to another one..., and if you don't come out, open a separate Thread in the correct Sub-Forum (=> 'Installation & Licensing' Sub-Forum), and @TechSup will be able to help you...
Your iMB v12.5 "Full" License should also be valid on FF/CR/IE to give you the 'PE' (Full) Version for those 3 Browsers, and even on IE, the v12.6 'Free' Version actually already supports the '!DATASOURCE' Command that you are using, so that could also be an "Option", even if you don't manage to migrate/import your License... Hum..., I'm not completely sure about the 'SAVEAS TYPE=EXTRACT' Command..., I think it is also supported in v12.6 'Free' for IE..., but I'm not "completely sure", you'd need to test/try... But those 2 Commands definitely don't work in v10.0.2 for FF 'Free' + v10.0.5 for CR 'Free', for those 2 Browsers, you then need the 'PE' Version...
>>>
OK, now "we go", ah-ah...!! Hum-hum..., OK, I understand your Scenario, hum..., quite Interesting actually, and I "also" have a very "shitty" Internet Connection, well... for other Reasons, but I'm a bit of a "Specialist" in terms of "Reliability" with Scripts and iMacros, ah-ah...! But you are "in good Hands", I'm "afraid", re-ah-ah...!
Alright, you'll need to implement a few Conditional "Mechanisms" to your Script, and to redo a Line/Row in your DataSource, you cannot use the "raw" '!LOOP' Var for '!DATASOURCE_LINE' anymore for '{{!COL1}}' and '{{!COL2}}' but '!DATASOURCE_LINE' will need to pull from some 2nd/secondary mini/Temp-DataSource which Row to use...
For that mini/Temp secondary DataSource, you'll need to use only 1 or 2 Rows in it, and to re-delete and re-create it on every Loop as 'SAVEAS' always creates (a) new Row(s) otherwise (appended) and finding or keeping track of the last Row to use is not very "straightforward"... (It can be done, I've already provided a Solution, but pfff..., will be "easier" for you if you don't go this Way, I would think..., just saying..., ah-ah...!)
For your "main" = current 'SAVEAS', you could "decide" if you want to keep it "like it currently is", => with some "#EANF#" Data in it, or if you maybe want to double that Process, one 'SAVEAS' saving all the Data for every Loop, that could serve as "Logging", and one 'SAVEAS' saving "only" the "clean"/usable Data, search the Forum for "conditional+saveas", I've already provided the Solution for that Func...
=> You basically need to use 'EVAL()' (about to become your "Best Friend", ah-ah...!) to dynamically compute the FileName to use for the 'SAVEAS'. And if you want to double the 'SAVEAS', you'll also need to keep the Content of '!EXTRACT' in some Temp Var as the first 'SAVEAS' will clear (by Design) the Content of '!EXTRACT' and the 2nd 'SAVEAS' would otherwise always save nothing...
>
All your 'WAIT' Statements, meant to slow your Script I understand, are not very "efficient"... I would rather keep only one 'WAIT' Statement, that can also be made Conditional when your IP is changing and the 'EXTRACT' returns "#EANF#", again "conditional+wait" will give you the Solution if you search the Forum on those 2 Terms...
Some "Interesting" Idea maybe, would be to also use your Temp secondary DataSource to keep track of the current + previous Value for the 'WAIT', to make that 'WAIT' Statement "double" Conditional, the first time you get some "#EANF#" in the 'EXTRACT', make it 5 Sec for example, then 10/15/20 Sec on the next Loops, until a Max of 20 or 30 Sec, until 'EXTRACT' works again and resets the 'WAIT' to 0/1/5, whatever Value you want to use when the Script "works fine" and reliably...
>
And I would think you don't "really" need the 2 'URL GOTO' Statements, both at the Beg + End of your Script, the first one at the Beg of your Script is "enough" I would think...
>
OK, "Good Luck" further, and post your "Final" Script, or dare to shout if you didn't understand anything or if you get stuck somewhere...
Oh...!, and I've lost my Co. of course...! OK, I'll try posting that Reply a bit "later"...
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
-
- Posts: 5
- Joined: Mon Aug 03, 2020 10:36 pm
Re: Trying to redo loop if EANF
Thank you for the post, Im going to go through this and try to muddle my way through some additional lines, the conditional wait sounds like itll be me entire solution actually. Could you elaborate on the ip checking functionality, that likely could solve my issues as well.
I was using IE as the browser and didn't have the free edition popups there but thought possibly it was the browser that was causing a lot of the issues so i am now trying to use Imacros browser but it seems the license doesn't automatically transfer so i need to hunt down the Key somewhere.
I was using IE as the browser and didn't have the free edition popups there but thought possibly it was the browser that was causing a lot of the issues so i am now trying to use Imacros browser but it seems the license doesn't automatically transfer so i need to hunt down the Key somewhere.
Re: Trying to redo loop if EANF
NotguyinVideo wrote: ↑Wed Aug 05, 2020 2:10 amThank you for the post, Im going to go through this and try to muddle my way through some additional lines, the conditional wait sounds like itll be me entire solution actually. Could you elaborate on the ip checking functionality, that likely could solve my issues as well.
I was using IE as the browser and didn't have the free edition popups there but thought possibly it was the browser that was causing a lot of the issues so i am now trying to use Imacros browser but it seems the license doesn't automatically transfer so i need to hunt down the Key somewhere.
Hum, "muddle my way" sounds like the "funny" beginning of some "exciting" Adventure, ah-ah...!
"the conditional wait sounds like itll be me entire solution actually..."
=> Hum, yep, why not, that would indeed be the "dirty but easy/iest" Solution, with some hard-coded and constant 5-7-10 Sec for the 'WAIT', when 'EXTRACT' returns "#EANF#", if the Data from your DataSource is "reliable" and you know for sure that "#EANF#" from the 'EXTRACT' means that your IP is changing at that time...
The Implementation is fairly easy, => stg like the following for example...:
Code: Select all
SET !EXTRACT NULL
TAG POS=1 TYPE=DIV ATTR=CLASS:"output" EXTRACT=TXT
SET Cond_WAIT EVAL("var s='{{!EXTRACT}}'; var z; if(s=='#EANF#'){z=10;} else{z=1;}; z;")
'PROMPT EXTRACT:<SP>_{{!EXTRACT}}_<BR><BR>Cond_WAIT:<SP>_{{Cond_WAIT}}_
WAIT SECONDS={{Cond_WAIT}}
>>>
About the Popups, only iMB has a 'Trial' Version and triggers those Popups, the 3 other Browsers (=> IE/FF/CR) have a 'PE' (= 'Personal Edition' = "Full") Version, and a 'Free' Version, but they don't trigger any Popup(s)...
User-Defined Vars are for example not supported in the 'Free' Versions, you would then need to change my "Cond_WAIT" Var to "!VAR1", as only '!VAR[1-3]' are supported as Names for Vars in those 'Free' Versions...
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
-
- Posts: 5
- Joined: Mon Aug 03, 2020 10:36 pm
Re: Trying to redo loop if EANF
So i figured first things first fix the license issue, So i got my chrome IIM working and licensed and it seems to work just fine now with no crashes. Im letting it run now for a few hours and we shall see but my initial basic coding may work just fine in chrome with no crash. WOOHOOOO
Stand by for my post later today where im like nah i was wrong.....
Stand by for my post later today where im like nah i was wrong.....
Re: Trying to redo loop if EANF
NotguyinVideo wrote: ↑Wed Aug 05, 2020 3:52 pmSo i figured first things first fix the license issue, So i got my chrome IIM working and licensed and it seems to work just fine now with no crashes. Im letting it run now for a few hours and we shall see but my initial basic coding may work just fine in chrome with no crash. WOOHOOOO
Stand by for my post later today where im like nah i was wrong.....
OK, Thanks for the Follow-up and the Update...
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
-
- Posts: 5
- Joined: Mon Aug 03, 2020 10:36 pm
Re: Trying to redo loop if EANF
Seems to run pretty good. I think I’ll follow up for some optimizations but I think most of my issue is internet explorer bugginess
Re: Trying to redo loop if EANF
NotguyinVideo wrote: ↑Wed Aug 05, 2020 5:25 pmSeems to run pretty good. I think I’ll follow up for some optimizations but I think most of my issue is internet explorer bugginess
OK, good to hear then...
But I still gave you some "pretty good" Advice anyway, about Logging and Conditional 'WAIT' to increase the "Reliability" of your Script...
I actually became a bit of a "Specialist" or "Advanced" User with iMacros for a very big part because of "unreliable" Internet Connection, always having to find some "Creative" Solution/Workaround for Failing Connection..., and now I can find some "Solution" for nearly every Scenario some Thread gets opened on the Forum, ah-ah...!
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...