Dynamic Frame Tracking

Discussions and Tech Support related to website data extraction, screen scraping and data mining using iMacros.

Moderators: Community Moderators, iMacros Moderators

Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the Google search box (at the top of each forum page) to see if a similar problem or question has already been addressed. This will search the entire contents of the forums as well as the iMacros Wiki.
3. We can respond much faster to your posts if you include the following information:

CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST

Answering your own posts (e.g. attempting to "bump" your topic) drops your topic from the list of unanswered threads, so it may actually receive less views.

Dynamic Frame Tracking

by azbob on Sat May 27, 2017 11:12 pm

Configuration: Surface Pro 2, Win 10, Firefox 53, iMacros Standard Edition (x86) Version 11.0.246.4051
Hi,
I extract data from a web page and in the last few months the frame number (no frame names that refer to table) seems to vary from numbers 14-18 every few days/weeks. My work around now is to manually change number. I would like to able to have macro handle that, but I can't figure out how to do it. The original macro has 300 lines of code and I need to access this particular frame twice, line 47 and then again at line 234. Ideally the macro would determine frame number, then store in variable for second access. I have searched the web and I know there are scripting solutions out there but I am confused/unsure how to specifically code and then incorporate script into the existing macro: ie: Run firstpart.iim, run framenumber.js, then run finalpart.iim??
Thanks

TEST Macro:
VERSION BUILD=11.0.246.4051
SET !ERRORIGNORE YES
SET !EXTRACT_TEST_POPUP YES
TAB OPEN
TAB T=2
URL GOTO=http://armls.flexmls.com/
FRAME NAME=top_frame
TAG POS=2 TYPE=A ATTR=TXT:Map<SP>search
FRAME NAME=view_frame
TAG POS=1 TYPE=SELECT ATTR=NAME:sdlist CONTENT=%x'20080728221329913527000000'
TAG POS=1 TYPE=BUTTON:SUBMIT ATTR=TXT:Use
TAG POS=1 TYPE=SELECT ATTR=NAME:s_3 CONTENT=%A:%AWC_C:%AWC_Y:%P:%C
TAG POS=1 TYPE=SPAN ATTR=ID:3_see_all_link
TAG POS=1 TYPE=INPUT:CHECKBOX ATTR=NAME:c_3_4_date CONTENT=YES
TAG POS=1 TYPE=INPUT:TEL ATTR=NAME:relative_3_4 CONTENT=365
TAG POS=1 TYPE=INPUT:CHECKBOX ATTR=ID:enabled_4 CONTENT=YES
FRAME NAME=iframe_map
TAG POS=1 TYPE=IMG ATTR=SRC:http://armls.flexmls.com/images/mapping/pin_24.png
FRAME NAME=view_frame
TAG POS=1 TYPE=SELECT ATTR=NAME:s_4 CONTENT=%SF
SET !VAR1 "4897 e hazeltine ct"
SET !VAR2 "85249"
SET !VAR7 "chandler"
'PROMPT "Please enter subject street address:" !VAR1
'PROMPT "Enter subject zip code:" !VAR2
'PROMPT "Enter subject city" !VAR7
FRAME NAME=iframe_map
TAG POS=1 TYPE=IMG ATTR=SRC:http://armls.flexmls.com/images/mapping/pin_24.png
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:NoFormName ATTR=NAME:address CONTENT={{!VAR1}}
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:NoFormName ATTR=NAME:zip CONTENT={{!VAR2}}
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:NoFormName ATTR=NAME:city CONTENT={{!VAR7}}
TAG POS=1 TYPE=BUTTON:SUBMIT ATTR=TXT:Locate
TAG POS=1 TYPE=DIV ATTR=ID:point_0
WAIT SECONDS=5
TAG POS=1 TYPE=SPAN ATTR=TXT:Use<SP>this<SP>location
TAG POS=1 TYPE=A ATTR=TXT:Radius<SP>Search
TAG POS=1 TYPE=A ATTR=TXT:Create<SP>Radius
WAIT SECONDS=10
FRAME NAME=view_frame
'Extract 12 month Neighborhood house price stats
TAG POS=1 TYPE=LI ATTR=ID:tab_stats
TAG POS=1 TYPE=A ATTR=TXT:Comparison<SP>Statistics
WAIT SECONDS=15
FRAME F=16
TAG POS=1 TYPE=TD ATTR=ID:hiLP EXTRACT=TXT
TAG POS=1 TYPE=TD ATTR=ID:ctLP EXTRACT=TXT
TAG POS=1 TYPE=TD ATTR=ID:lowSP EXTRACT=TXT
TAG POS=1 TYPE=TD ATTR=ID:hiSP EXTRACT=TXT
TAG POS=1 TYPE=SPAN ATTR=ID:ctSP EXTRACT=TXT
TAG POS=1 TYPE=TD ATTR=ID:avgDom EXTRACT=TXT
FILEDELETE NAME=C:\Users\Public\Documents\iMacros\datasources\BBformdata\Chasedata.csv
SAVEAS TYPE=EXTRACT FOLDER=C:\Users\Public\Documents\iMacros\datasources\BBformdata FILE=Chasedata.csv
azbob
 
Posts: 76
Joined: Mon Sep 21, 2009 11:16 am

Re: Dynamic Frame Tracking

by chivracq on Sun May 28, 2017 4:22 am

azbob wrote:Configuration:
Code: Select all
Surface Pro 2, Win 10, Firefox 53, iMacros Standard Edition (x86) Version 11.0.246.4051

Hi,
I extract data from a web page and in the last few months the frame number (no frame names that refer to table) seems to vary from numbers 14-18 every few days/weeks. My work around now is to manually change number. I would like to able to have macro handle that, but I can't figure out how to do it. The original macro has 300 lines of code and I need to access this particular frame twice, line 47 and then again at line 234. Ideally the macro would determine frame number, then store in variable for second access. I have searched the web and I know there are scripting solutions out there but I am confused/unsure how to specifically code and then incorporate script into the existing macro: ie: Run firstpart.iim, run framenumber.js, then run finalpart.iim??
Thanks

TEST Macro:
Code: Select all
VERSION BUILD=11.0.246.4051
SET !ERRORIGNORE YES
SET !EXTRACT_TEST_POPUP YES
TAB OPEN
TAB T=2
URL GOTO=http://armls.flexmls.com/
FRAME NAME=top_frame
TAG POS=2 TYPE=A ATTR=TXT:Map<SP>search
FRAME NAME=view_frame
TAG POS=1 TYPE=SELECT ATTR=NAME:sdlist CONTENT=%x'20080728221329913527000000'
TAG POS=1 TYPE=BUTTON:SUBMIT ATTR=TXT:Use
TAG POS=1 TYPE=SELECT ATTR=NAME:s_3 CONTENT=%A:%AWC_C:%AWC_Y:%P:%C
TAG POS=1 TYPE=SPAN ATTR=ID:3_see_all_link
TAG POS=1 TYPE=INPUT:CHECKBOX ATTR=NAME:c_3_4_date CONTENT=YES
TAG POS=1 TYPE=INPUT:TEL ATTR=NAME:relative_3_4 CONTENT=365
TAG POS=1 TYPE=INPUT:CHECKBOX ATTR=ID:enabled_4 CONTENT=YES
FRAME NAME=iframe_map
TAG POS=1 TYPE=IMG ATTR=SRC:http://armls.flexmls.com/images/mapping/pin_24.png
FRAME NAME=view_frame
TAG POS=1 TYPE=SELECT ATTR=NAME:s_4 CONTENT=%SF
SET !VAR1 "4897 e hazeltine ct"
SET !VAR2 "85249"
SET !VAR7 "chandler"
'PROMPT "Please enter subject street address:" !VAR1
'PROMPT "Enter subject zip code:" !VAR2
'PROMPT "Enter subject city" !VAR7
FRAME NAME=iframe_map
TAG POS=1 TYPE=IMG ATTR=SRC:http://armls.flexmls.com/images/mapping/pin_24.png
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:NoFormName ATTR=NAME:address CONTENT={{!VAR1}}
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:NoFormName ATTR=NAME:zip CONTENT={{!VAR2}}
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:NoFormName ATTR=NAME:city CONTENT={{!VAR7}}
TAG POS=1 TYPE=BUTTON:SUBMIT ATTR=TXT:Locate
TAG POS=1 TYPE=DIV ATTR=ID:point_0
WAIT SECONDS=5
TAG POS=1 TYPE=SPAN ATTR=TXT:Use<SP>this<SP>location
TAG POS=1 TYPE=A ATTR=TXT:Radius<SP>Search
TAG POS=1 TYPE=A ATTR=TXT:Create<SP>Radius
WAIT SECONDS=10
FRAME NAME=view_frame
'Extract 12 month Neighborhood house price stats
TAG POS=1 TYPE=LI ATTR=ID:tab_stats
TAG POS=1 TYPE=A ATTR=TXT:Comparison<SP>Statistics
WAIT SECONDS=15
FRAME F=16
TAG POS=1 TYPE=TD ATTR=ID:hiLP EXTRACT=TXT
TAG POS=1 TYPE=TD ATTR=ID:ctLP EXTRACT=TXT
TAG POS=1 TYPE=TD ATTR=ID:lowSP EXTRACT=TXT
TAG POS=1 TYPE=TD ATTR=ID:hiSP EXTRACT=TXT
TAG POS=1 TYPE=SPAN ATTR=ID:ctSP EXTRACT=TXT
TAG POS=1 TYPE=TD ATTR=ID:avgDom EXTRACT=TXT
FILEDELETE NAME=C:\Users\Public\Documents\iMacros\datasources\BBformdata\Chasedata.csv
SAVEAS TYPE=EXTRACT FOLDER=C:\Users\Public\Documents\iMacros\datasources\BBformdata FILE=Chasedata.csv

If your Frame_Nb varies within [14,18] => 5 Values, "my" Solution would be to 5 times "try" to extract one (or more) of the Fields you want to extract to check if it is present in that Frame_Nb to therefore "deduct" which Frame_Nb to use using 'EVAL()' + 'CASE' or 'if else' for example, stg like:
Code: Select all
'Check FRAME_Nb for F[14,18]:
FRAME F=14
SET !EXTRACT NULL
TAG POS=1 TYPE=TD ATTR=ID:hiLP EXTRACT=TXT
SET F_14 {{!EXTRACT}}
'>
FRAME F=15
SET !EXTRACT NULL
TAG POS=1 TYPE=TD ATTR=ID:hiLP EXTRACT=TXT
SET F_15 {{!EXTRACT}}
'>
FRAME F=16
SET !EXTRACT NULL
TAG POS=1 TYPE=TD ATTR=ID:hiLP EXTRACT=TXT
SET F_16 {{!EXTRACT}}
'>
FRAME F=17
SET !EXTRACT NULL
TAG POS=1 TYPE=TD ATTR=ID:hiLP EXTRACT=TXT
SET F_17 {{!EXTRACT}}
'>
FRAME F=18
SET !EXTRACT NULL
TAG POS=1 TYPE=TD ATTR=ID:hiLP EXTRACT=TXT
SET F_18 {{!EXTRACT}}

'Deduct FRAME_Nb:
SET FRAME_Nb EVAL("var f14='{{F_14}}', f15='{{F_15}}', f16='{{F_16}}', f17='{F_17}}', f18='{{F_18}}'; var nf='#EANF#'; var z; if(f14!=nf){z=14;} else if(f15!=nf){z=15;} else if(f16!=nf){z=16;} else if(f17!=nf){z=17;} else if(f18!=nf){z=18;} else{z='ERROR...!';}; z;")
PROMPT FRAME_Nb:<SP>_{{FRAME_Nb}}_
PAUSE

FRAME F={{FRAME_Nb}}
SET !EXTRACT NULL
TAG POS=1 TYPE=TD ATTR=ID:hiLP EXTRACT=TXT
TAG POS=1 TYPE=TD ATTR=ID:ctLP EXTRACT=TXT
TAG POS=1 TYPE=TD ATTR=ID:lowSP EXTRACT=TXT
TAG POS=1 TYPE=TD ATTR=ID:hiSP EXTRACT=TXT
TAG POS=1 TYPE=SPAN ATTR=ID:ctSP EXTRACT=TXT
TAG POS=1 TYPE=TD ATTR=ID:avgDom EXTRACT=TXT
Not tested as your Site is behind Login&Password but I guess you can understand the "Principle"...

The 5x 'if else' Construction is maybe not very "elegant", there are probably more elegant Solutions, ah-ah...!, but this one should work... :oops:
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all 3 Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
chivracq
 
Posts: 6294
Joined: Sat Apr 13, 2013 6:07 am
Location: Amsterdam (NL)

Re: Dynamic Frame Tracking

by azbob on Mon May 29, 2017 1:16 am

Hi,
That works.
Great.
Thanks.

While waiting I tried a different approach. A JS loop that calls macro. However, I get an undefined on the prompt "framenumber"
Any ideas why? Is it not passing thru to macro or is the loop not working?

Pre loop macro:
VERSION BUILD=11.0.246.4051
SET !ERRORIGNORE YES
SET !EXTRACT_TEST_POPUP YES
TAB OPEN
TAB T=2
URL GOTO=http://armls.flexmls.com/
FRAME NAME=top_frame
TAG POS=2 TYPE=A ATTR=TXT:Map<SP>search
FRAME NAME=view_frame
TAG POS=1 TYPE=SELECT ATTR=NAME:sdlist CONTENT=%x'20080728221329913527000000'
TAG POS=1 TYPE=BUTTON:SUBMIT ATTR=TXT:Use
TAG POS=1 TYPE=SELECT ATTR=NAME:s_3 CONTENT=%A:%AWC_C:%AWC_Y:%P:%C
TAG POS=1 TYPE=SPAN ATTR=ID:3_see_all_link
TAG POS=1 TYPE=INPUT:CHECKBOX ATTR=NAME:c_3_4_date CONTENT=YES
TAG POS=1 TYPE=INPUT:TEL ATTR=NAME:relative_3_4 CONTENT=365
TAG POS=1 TYPE=INPUT:CHECKBOX ATTR=ID:enabled_4 CONTENT=YES
FRAME NAME=iframe_map
TAG POS=1 TYPE=IMG ATTR=SRC:http://armls.flexmls.com/images/mapping/pin_24.png
FRAME NAME=view_frame
TAG POS=1 TYPE=SELECT ATTR=NAME:s_4 CONTENT=%SF
SET !VAR1 "4897 e hazeltine ct"
SET !VAR2 "85249"
SET !VAR7 "chandler"
'PROMPT "Please enter subject street address:" !VAR1
'PROMPT "Enter subject zip code:" !VAR2
'PROMPT "Enter subject city" !VAR7
FRAME NAME=iframe_map
TAG POS=1 TYPE=IMG ATTR=SRC:http://armls.flexmls.com/images/mapping/pin_24.png
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:NoFormName ATTR=NAME:address CONTENT={{!VAR1}}
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:NoFormName ATTR=NAME:zip CONTENT={{!VAR2}}
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:NoFormName ATTR=NAME:city CONTENT={{!VAR7}}
TAG POS=1 TYPE=BUTTON:SUBMIT ATTR=TXT:Locate
TAG POS=1 TYPE=DIV ATTR=ID:point_0
WAIT SECONDS=5
TAG POS=1 TYPE=SPAN ATTR=TXT:Use<SP>this<SP>location
TAG POS=1 TYPE=A ATTR=TXT:Radius<SP>Search
TAG POS=1 TYPE=A ATTR=TXT:Create<SP>Radius
WAIT SECONDS=10
FRAME NAME=view_frame
'Extract 12 month Neighborhood house price stats
TAG POS=1 TYPE=LI ATTR=ID:tab_stats
TAG POS=1 TYPE=A ATTR=TXT:Comparison<SP>Statistics
WAIT SECONDS=10

JS LOOP Scection:

iimPlay('++AATEST_PRE_LOOP.iim');
var find_number = "CODE:";
find_number += "FRAME F={{i}}" + "\n";
find_number += "TAG POS=1 TYPE=TD ATTR=ID:hiLP EXTRACT=TXT" + "\n";

//zero out framenumber
framenumber=0;

//Start at frame 14 and go to 20
for(i=14;i<=20;i++)
{
//Set variable to current frame number pass to post loop macro
iimSet("framenumber",i);

//if it can extract(true) than save frame number and play post loop macro
if(iimPlay(find_number)==true)

{
//Store current framenumber Is this redundant??
framenumber=i;

}
}
iimPlay('++AATEST_POST_LOOP.iim');

Post loop macro:
PROMPT {{framenumber}}
FRAME F={{framenumber}}
TAG POS=1 TYPE=TD ATTR=ID:hiLP EXTRACT=TXT
TAG POS=1 TYPE=TD ATTR=ID:ctLP EXTRACT=TXT
TAG POS=1 TYPE=TD ATTR=ID:lowSP EXTRACT=TXT
TAG POS=1 TYPE=TD ATTR=ID:hiSP EXTRACT=TXT
TAG POS=1 TYPE=SPAN ATTR=ID:ctSP EXTRACT=TXT
TAG POS=1 TYPE=TD ATTR=ID:avgDom EXTRACT=TXT
FILEDELETE NAME=C:\Users\Public\Documents\iMacros\datasources\BBformdata\Chasedata.csv
SAVEAS TYPE=EXTRACT FOLDER=C:\Users\Public\Documents\iMacros\datasources\BBformdata FILE=Chasedata.csv

Thanks,
Bob
azbob
 
Posts: 76
Joined: Mon Sep 21, 2009 11:16 am

Re: Dynamic Frame Tracking

by chivracq on Mon May 29, 2017 9:51 am

azbob wrote:Hi,
That works.
Great.
Thanks.

OK, good to hear...

The same "Technique" can be used for different Tabs in the Browser as well or even Pages on a same Site if the Number of Occurrences to check remains a bit limited like the 5 Frame Numbers in your Case... Otherwise you might need to switch to a Solution using a '.js' Script like you do...

azbob wrote:While waiting I tried a different approach. A JS loop that calls macro. However, I get an undefined on the prompt "framenumber"
Any ideas why? Is it not passing thru to macro or is the loop not working?

Pre loop macro:
Code: Select all
VERSION BUILD=11.0.246.4051
SET !ERRORIGNORE YES
SET !EXTRACT_TEST_POPUP YES
TAB OPEN
TAB T=2
URL GOTO=http://armls.flexmls.com/
FRAME NAME=top_frame
TAG POS=2 TYPE=A ATTR=TXT:Map<SP>search
FRAME NAME=view_frame
TAG POS=1 TYPE=SELECT ATTR=NAME:sdlist CONTENT=%x'20080728221329913527000000'
TAG POS=1 TYPE=BUTTON:SUBMIT ATTR=TXT:Use
TAG POS=1 TYPE=SELECT ATTR=NAME:s_3 CONTENT=%A:%AWC_C:%AWC_Y:%P:%C
TAG POS=1 TYPE=SPAN ATTR=ID:3_see_all_link
TAG POS=1 TYPE=INPUT:CHECKBOX ATTR=NAME:c_3_4_date CONTENT=YES
TAG POS=1 TYPE=INPUT:TEL ATTR=NAME:relative_3_4 CONTENT=365
TAG POS=1 TYPE=INPUT:CHECKBOX ATTR=ID:enabled_4 CONTENT=YES
FRAME NAME=iframe_map
TAG POS=1 TYPE=IMG ATTR=SRC:http://armls.flexmls.com/images/mapping/pin_24.png
FRAME NAME=view_frame
TAG POS=1 TYPE=SELECT ATTR=NAME:s_4 CONTENT=%SF
SET !VAR1 "4897 e hazeltine ct"
SET !VAR2 "85249"
SET !VAR7 "chandler"
'PROMPT "Please enter subject street address:" !VAR1
'PROMPT "Enter subject zip code:" !VAR2
'PROMPT "Enter subject city" !VAR7
FRAME NAME=iframe_map
TAG POS=1 TYPE=IMG ATTR=SRC:http://armls.flexmls.com/images/mapping/pin_24.png
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:NoFormName ATTR=NAME:address CONTENT={{!VAR1}}
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:NoFormName ATTR=NAME:zip CONTENT={{!VAR2}}
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:NoFormName ATTR=NAME:city CONTENT={{!VAR7}}
TAG POS=1 TYPE=BUTTON:SUBMIT ATTR=TXT:Locate
TAG POS=1 TYPE=DIV ATTR=ID:point_0
WAIT SECONDS=5
TAG POS=1 TYPE=SPAN ATTR=TXT:Use<SP>this<SP>location
TAG POS=1 TYPE=A ATTR=TXT:Radius<SP>Search
TAG POS=1 TYPE=A ATTR=TXT:Create<SP>Radius
WAIT SECONDS=10
FRAME NAME=view_frame
'Extract 12 month Neighborhood house price stats
TAG POS=1 TYPE=LI ATTR=ID:tab_stats
TAG POS=1 TYPE=A ATTR=TXT:Comparison<SP>Statistics
WAIT SECONDS=10


JS LOOP Scection:
Code: Select all
iimPlay('++AATEST_PRE_LOOP.iim');
var find_number =  "CODE:";
find_number +=  "FRAME F={{i}}" + "\n";
find_number +=  "TAG POS=1 TYPE=TD ATTR=ID:hiLP EXTRACT=TXT" + "\n";

//zero out framenumber
framenumber=0;

//Start at frame 14 and go to 20
for(i=14;i<=20;i++)
{
//Set variable to current frame number pass to post loop macro
iimSet("framenumber",i);

//if it can extract(true) than save frame number and play post loop macro
if(iimPlay(find_number)==true)

{
//Store current framenumber Is this redundant??
framenumber=i;

}
}
iimPlay('++AATEST_POST_LOOP.iim');


Post loop macro:
Code: Select all
PROMPT {{framenumber}}
FRAME F={{framenumber}}
TAG POS=1 TYPE=TD ATTR=ID:hiLP EXTRACT=TXT
TAG POS=1 TYPE=TD ATTR=ID:ctLP EXTRACT=TXT
TAG POS=1 TYPE=TD ATTR=ID:lowSP EXTRACT=TXT
TAG POS=1 TYPE=TD ATTR=ID:hiSP EXTRACT=TXT
TAG POS=1 TYPE=SPAN ATTR=ID:ctSP EXTRACT=TXT
TAG POS=1 TYPE=TD ATTR=ID:avgDom EXTRACT=TXT
FILEDELETE NAME=C:\Users\Public\Documents\iMacros\datasources\BBformdata\Chasedata.csv
SAVEAS TYPE=EXTRACT FOLDER=C:\Users\Public\Documents\iMacros\datasources\BBformdata FILE=Chasedata.csv


Thanks,
Bob

Hum..., try using the ]CODE[ Forum Meta-Tags with Scripts (like I do in my Quotes...), Posts and Threads become quickly difficult to read otherwise with lengthy Posts/Scripts... :idea:

Euuuuuuhhh..., yep..., dunno, I don't do '.js' Scripts, ah-ah...!, but your "if(iimPlay(find_number)==true)" looks incorrect to me, I would think..., as 'iimPlay()' returns a Number, "1" if the Macro was successful and some ReturnCode <0 if any Error occurred, and I'm not sure exactly about the "influence" of '!ERRORIGNORE' and 'EXTRACT' on the ReturnCode for a Macro, you would need to test that...
But it will never return 'TRUE', I would think...

And maybe even more "precise" and reliable than the ReturnCode could be to use 'iimGetExtract()' a bit like I did in my 'EVAL()' Statement to check the Content of the '!EXTRACT' Var if the Field was found or not... :idea:
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all 3 Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
chivracq
 
Posts: 6294
Joined: Sat Apr 13, 2013 6:07 am
Location: Amsterdam (NL)

Re: Dynamic Frame Tracking

by azbob on Mon May 29, 2017 5:05 pm

Hi,
Great suggestions. I'll give it a try and let you know.
Thanks,
Bob
azbob
 
Posts: 76
Joined: Mon Sep 21, 2009 11:16 am

Re: Dynamic Frame Tracking

by chivracq on Tue May 30, 2017 1:15 am

azbob wrote:Hi,
Great suggestions. I'll give it a try and let you know.
Thanks,
Bob

Well..., my "great suggestion" about editing your Posts with "try using the ]CODE[ Forum Meta-Tags with Scripts" shouldn't be very hard to implement, ah-ah...!
Good luck for the rest and post your final Script indeed...
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all 3 Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
chivracq
 
Posts: 6294
Joined: Sat Apr 13, 2013 6:07 am
Location: Amsterdam (NL)

Re: Dynamic Frame Tracking

by azbob on Mon Jun 12, 2017 12:12 am

Configuration: Surface Pro 2, Win 10, Firefox 53, iMacros Standard Edition (x86) Version 11.0.246.4051
Hi again,
Well I have been working to come up with a more "elegant" :) JS solution to find the frame number. I am almost there but for a nagging problem. In this case the correct frame number for this week is 16, although it can vary from 14-18 for testing I set the loop to 15-17.
The macro statement in line 15 in the JS loop doesn't seem to register to the extract statements. As it returns #EANF# with all three frame numbers, the loop cycles through ends at 17 and that is passed to POS_LOOP.
However if I comment out line 15(//), un-comment line 14, set loop start to 16(line 6) and end to 16(line 19). In other words by passing for-loop and forcing the frame to number 16 the extract returns $449,000(meaning I have correct frame number), passes the frame number to POST_LOOP macro. POST_LOOP proceeds to extract all info needed from page.
For a while I was getting this error message "BadParameter: expected F=<number> as parameter 1, line 2 (Error code: -911)".
So I added line 6 and line 21 to force a number and set !errorignore to YES.
I have forced loop to print all variables to screen as well as PROMPTS in POST_LOOP.
PRE_LOOP gets me to web page,JS loop finds the "working" frame number and then passes that to POS_LOOP macro for the extraction.
Soooo why does it not work when I use a variable?

Thanks and good hunting!
Code: Select all
iimPlay('++AATEST_PRE_LOOP.iim');
//declarations
var framenumber;
var retcode;
//var Frame_nb;
var fnb=(15*1)

var find_number =  "CODE:";
find_number += "SET !ERRORIGNORE YES" + "\n";
find_number += "SET !EXTRACT NULL" + "\n";
//find_number += "SET !VAR0 {{i}}" + "\n";
//find_number +=  "FRAME F={{i}}" + "\n";
//find_number += "SET fnb EVAL("var fn='{{fnb}}'; var z; z=('{{fn}}'*1); z;")" + "\n";
//find_number +=  "FRAME F=16" + "\n";
find_number +=  "FRAME F={{Frame_nb}}" + "\n";
find_number +=  "TAG POS=1 TYPE=TD ATTR=TXT:High" + "\n";
find_number +=  "TAG POS=R1 TYPE=TD ATTR=ID:hiLP EXTRACT=TXT" + "\n";

for(; fnb <= 17; fnb++){
   alert(fnb);
   var Frame_nb = eval(fnb*1);
   retcode=iimPlay(find_number);
   var frame_valid = iimGetLastExtract(1);
   alert(iimGetLastExtract(1));
   alert(Frame_nb);
   alert(frame_valid);
   //var frame_valid = iimGetExtract();
   iimSet("framenumber",fnb);
   iimSet("RetCode",retcode);
   iimSet("extract",frame_valid);

//if extract was frame number than save frame number and play rest of macro
//if(retcode => 0)
   if (frame_valid !== "#EANF#")
//if (!(frame_valid===null) || !(frame_valid===''))
     
 //      {frame_valid != "#EANF#"}
//document.write (frame_valid);
   {
   var framenumber=fnb;
   break;
   }
   
   }
   
iimPlay('++AATEST_POST_LOOP.iim');

[img]
Frame%20number%20Using%20variable.jpg
[/img]
Attachments
Frame number Using variable.jpg
Web page, tagged fields and Prompt readout
azbob
 
Posts: 76
Joined: Mon Sep 21, 2009 11:16 am

Re: Dynamic Frame Tracking

by chivracq on Mon Jun 12, 2017 4:20 pm

azbob wrote:Configuration:
Code: Select all
Surface Pro 2, Win 10, Firefox 53, iMacros Standard Edition (x86) Version 11.0.246.4051

Hi again,
Well I have been working to come up with a more "elegant" :) JS solution to find the frame number. I am almost there but for a nagging problem. In this case the correct frame number for this week is 16, although it can vary from 14-18 for testing I set the loop to 15-17.
The macro statement in line 15 in the JS loop doesn't seem to register to the extract statements. As it returns #EANF# with all three frame numbers, the loop cycles through ends at 17 and that is passed to POS_LOOP.
However if I comment out line 15(//), un-comment line 14, set loop start to 16(line 6) and end to 16(line 19). In other words by passing for-loop and forcing the frame to number 16 the extract returns $449,000(meaning I have correct frame number), passes the frame number to POST_LOOP macro. POST_LOOP proceeds to extract all info needed from page.
For a while I was getting this error message "BadParameter: expected F=<number> as parameter 1, line 2 (Error code: -911)".
So I added line 6 and line 21 to force a number and set !errorignore to YES.
I have forced loop to print all variables to screen as well as PROMPTS in POST_LOOP.
PRE_LOOP gets me to web page,JS loop finds the "working" frame number and then passes that to POS_LOOP macro for the extraction.
Soooo why does it not work when I use a variable?

Thanks and good hunting!
Code: Select all
iimPlay('++AATEST_PRE_LOOP.iim');
//declarations
var framenumber;
var retcode;
//var Frame_nb;
var fnb=(15*1)

var find_number =  "CODE:";
find_number += "SET !ERRORIGNORE YES" + "\n";
find_number += "SET !EXTRACT NULL" + "\n";
//find_number += "SET !VAR0 {{i}}" + "\n";
//find_number +=  "FRAME F={{i}}" + "\n";
//find_number += "SET fnb EVAL("var fn='{{fnb}}'; var z; z=('{{fn}}'*1); z;")" + "\n";
//find_number +=  "FRAME F=16" + "\n";
find_number +=  "FRAME F={{Frame_nb}}" + "\n";
find_number +=  "TAG POS=1 TYPE=TD ATTR=TXT:High" + "\n";
find_number +=  "TAG POS=R1 TYPE=TD ATTR=ID:hiLP EXTRACT=TXT" + "\n";

for(; fnb <= 17; fnb++){
   alert(fnb);
   var Frame_nb = eval(fnb*1);
   retcode=iimPlay(find_number);
   var frame_valid = iimGetLastExtract(1);
   alert(iimGetLastExtract(1));
   alert(Frame_nb);
   alert(frame_valid);
   //var frame_valid = iimGetExtract();
   iimSet("framenumber",fnb);
   iimSet("RetCode",retcode);
   iimSet("extract",frame_valid);

//if extract was frame number than save frame number and play rest of macro
//if(retcode => 0)
   if (frame_valid !== "#EANF#")
//if (!(frame_valid===null) || !(frame_valid===''))
     
 //      {frame_valid != "#EANF#"}
//document.write (frame_valid);
   {
   var framenumber=fnb;
   break;
   }
   
   }
   
iimPlay('++AATEST_POST_LOOP.iim');

[img]
Frame%20number%20Using%20variable.jpg
[/img]

Oh..., good...!, you finally found the ]CODE[ Tags, I was getting tired of asking..., you could still apply those to your previous Posts in this Thread...

But, pfff..., difficult to debug your Script if I cannot test it, I need to run it "mentally" in my Head, and I hardly know anything about JS as I never use '.js' Scripts, so it's a bit difficult to debug it... Use 'PROMPT' in your on-the-fly Macro to follow your Vars... (and disable '!ERRORIGNORE' or you don't get any relevant RuntimeErrors...).
=> I notice for example that you try to use some 'i' Var which comes out of nowhere while you define it as 'framenumber' in the '.js' Script but you then try to define a Var called 'fnb' in the '.iim' Script by referring to the same 'fnb' Var with the same name which doesn't exist yet and which uses as well a 'fn' Var which has not been defined, it's a complete "Mess", ah-ah...!
You then try to call it 'Frame_nb' which has not been defined...
=> Your Frame_Nb is Undefined, not wonder the 'EXTRACT' returns "#EANF#" each time...
I got lost, ah-ah...!

For only 5 possible Frame Nb's, simply use the Solution I gave you if it works, your "elegant" Solution is a bit too "dirty" to me, ah-ah...! :idea:
Last edited by chivracq on Tue Jun 13, 2017 6:26 am, edited 1 time in total.
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all 3 Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
chivracq
 
Posts: 6294
Joined: Sat Apr 13, 2013 6:07 am
Location: Amsterdam (NL)

Re: Dynamic Frame Tracking

by azbob on Mon Jun 12, 2017 7:06 pm

Okay I get it the other works fine, but now it is a challenge to solve why the Frame F= is not working. I removed the prior statements, which I commented out to give me a record of attempts. If you don't understand js can you pass it along to someone who does??
Thanks
Code: Select all
iimPlay('++AATEST_PRE_LOOP.iim');
//declarations
var framenumber;
var retcode;
var fnb=(15*1);

var find_number =  "CODE:";
find_number += "SET !ERRORIGNORE YES" + "\n";
find_number += "SET !EXTRACT NULL" + "\n";
find_number +=  "FRAME F={{Frame_nb}}" + "\n";
find_number +=  "TAG POS=1 TYPE=TD ATTR=TXT:High" + "\n";
find_number +=  "TAG POS=R1 TYPE=TD ATTR=ID:hiLP EXTRACT=TXT" + "\n";

for(; fnb <= 17; fnb++)
        {
   alert(fnb);
   var Frame_nb = eval(fnb*1);
   retcode=iimPlay(find_number);
   var frame_valid = iimGetLastExtract(1);
   alert(iimGetLastExtract(1));
   alert(Frame_nb);
   alert(frame_valid);
   iimSet("framenumber",fnb);
   iimSet("RetCode",retcode);
   iimSet("extract",frame_valid);

//if extract was frame number than save frame number and play rest of macro
   if (frame_valid !== "#EANF#")
   {
   var framenumber=fnb;
   break;
   }
   
   }
   
iimPlay('++AATEST_POST_LOOP.iim');
azbob
 
Posts: 76
Joined: Mon Sep 21, 2009 11:16 am

Re: Dynamic Frame Tracking

by chivracq on Tue Jun 13, 2017 8:58 am

azbob wrote:Okay I get it the other works fine, but now it is a challenge to solve why the Frame F= is not working. I removed the prior statements, which I commented out to give me a record of attempts. If you don't understand js can you pass it along to someone who does??
Thanks
Code: Select all
iimPlay('++AATEST_PRE_LOOP.iim');
//declarations
var framenumber;
var retcode;
var fnb=(15*1);

var find_number =  "CODE:";
find_number += "SET !ERRORIGNORE YES" + "\n";
find_number += "SET !EXTRACT NULL" + "\n";
find_number +=  "FRAME F={{Frame_nb}}" + "\n";
find_number +=  "TAG POS=1 TYPE=TD ATTR=TXT:High" + "\n";
find_number +=  "TAG POS=R1 TYPE=TD ATTR=ID:hiLP EXTRACT=TXT" + "\n";

for(; fnb <= 17; fnb++)
        {
   alert(fnb);
   var Frame_nb = eval(fnb*1);
   retcode=iimPlay(find_number);
   var frame_valid = iimGetLastExtract(1);
   alert(iimGetLastExtract(1));
   alert(Frame_nb);
   alert(frame_valid);
   iimSet("framenumber",fnb);
   iimSet("RetCode",retcode);
   iimSet("extract",frame_valid);

//if extract was frame number than save frame number and play rest of macro
   if (frame_valid !== "#EANF#")
   {
   var framenumber=fnb;
   break;
   }
   
   }
   
iimPlay('++AATEST_POST_LOOP.iim');

Hum, if it's for the "Challenge", then be it, ah-ah...! 8)
Well, don't worry, I'm no JS Guru but I can follow your Script even without being able to run it... And about "... can you pass it along to someone who does??", well, I'm only one User on this Forum, I don't have any "Influence" on other Users who might want to jump into your Thread...

But, OK, again..., use 'PROMPT' I told you to debug your Script in the '.iim' part, and like I already said, you are trying to use a 'Frame_nb' Var which has not been defined... :idea:
Have a look maybe at this Post for example with a "decent" Example of running an on-the-fly '.iim' Macro from a '.js' Script with (Nested) Loops...

>>>

Oh...!, and while "thinking" a bit about your Case, I think I found an easy and "elegant" if not "clever" way I guess to dynamically identify the Frame_Nb in just one fairly simple 'EVAL()' Statement in pure '.iim', ah-ah...! 8)

Find me an Example of a public Site/Page with (a variable or changing Nb of) multiple Frames (with or without ID, I won't use it), (or upload a (zipped) Full Saveas of your Page to the Thread...), and I can test/demonstrate this Method... (It is a bit based on "my Methods" to identify Dynamic Id's or the Max Nb of Options in a DDLB or my Workarounds for the 'SEARCH' Command and avoiding to use 'REGEX' that I have already posted + adding some kind of "ingredient" with "Relative Positioning" (which doesn't work "directly" with the 'FRAME' Command (unfortunately!)...) which is maybe not even necessary in your Case if the Frame you are interested in is always the last Frame on the Page... :idea:
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all 3 Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
chivracq
 
Posts: 6294
Joined: Sat Apr 13, 2013 6:07 am
Location: Amsterdam (NL)

Re: Dynamic Frame Tracking

by azbob on Tue Jun 13, 2017 11:20 am

Great..
A couple of things..
1. Line 21 defines variable Frame_nb
Code: Select all
var Frame_nb = eval(fnb*1);

2. I tested originally with " SET !ERRORIGNORE NO" and got two results:
a. It threw: "BadParameter: expected F=<number> as parameter 1, line 3 (Error code: -911)" after first run of extract at line 17 (Line 3 refers to third line in macro "Find_number" or actual line 15 in overall) so that is what got me thinking I had to force integer. So i put in line 6
Code: Select all
var: fnd=(15*1)

to set loop start frame number start point and again line 21:
Code: Select all
var Frame_nb=eval(fnb*1)

to force integer to each frame number after 15.

b. Extract returned "NULL" instead of #EANF# after each loop through Line 17 and at end it would throw "TypeError: val is null, line -439 (Error code: -991)"

3. Put this in Line 16 of iim:
Code: Select all
find_number +=  "PROMPT {{Frame_nb}} " = "\n" 

and it threw "ReferenceError: invalid assignment left-hand side, line 16 (Error code: -991)". So, apparently can't prompt with embedded iim in .js??
4. Zipped web page attached
BPO _ flexmls Web.zip
Zipped web page
(1.77 KIB) Downloaded 33 times


Onward and upward
azbob
 
Posts: 76
Joined: Mon Sep 21, 2009 11:16 am

Re: Dynamic Frame Tracking

by chivracq on Tue Jun 13, 2017 12:13 pm

azbob wrote:Great..
A couple of things..
1. Line 21 defines variable Frame_nb
Code: Select all
var Frame_nb = eval(fnb*1);

2. I tested originally with " SET !ERRORIGNORE NO" and got two results:
a. It threw: "BadParameter: expected F=<number> as parameter 1, line 3 (Error code: -911)" after first run of extract at line 17 (Line 3 refers to third line in macro "Find_number" or actual line 15 in overall) so that is what got me thinking I had to force integer. So i put in line 6
Code: Select all
var: fnd=(15*1)

to set loop start frame number start point and again line 21:
Code: Select all
var Frame_nb=eval(fnb*1)

to force integer to each frame number after 15.

b. Extract returned "NULL" instead of #EANF# after each loop through Line 17 and at end it would throw "TypeError: val is null, line -439 (Error code: -991)"

Yep, all this part is about the Var 'Frame_nb' not being defined in the '.iim' Macro...
Nice that you define a Var with the same name in your '.js', but the '.iim' Macro doesn't know anything about what happens in the '.js' part unless you declare a Var with the name that you use in the '.iim' Macro using 'iimSet()'..., which is still missing...

azbob wrote:3. Put this in Line 16 of iim:
Code: Select all
find_number +=  "PROMPT {{Frame_nb}} " = "\n" 

and it threw "ReferenceError: invalid assignment left-hand side, line 16 (Error code: -991)". So, apparently can't prompt with embedded iim in .js??

Hum..., don't know about that one, afaik, you can use 'PROMPT' in an on-the-fly '.iim' Macro, I think...
And if for any Reason, 'PROMPT' doesn't work, then you can use '!EXTRACT', the OS Clipboard or a Temp 'SAVEAS' to debug that part...

azbob wrote:4. Zipped web page attached
BPO _ flexmls Web.zip


Onward and upward

Yep, but I only get a blank Page as all related Files for the Full HTML Saveas are missing:
File not found

Pale Moon can't find the file at /D:/TEMP/iMacros/Temp/_Forum Cases/azbob/BPO _ flexmls Web/BPO _ flexmls Web_files/mainmenu.htm.

I can see a Ref to this 'mainmenu.htm' File in the Source Code of your Attachment, it is actually defined as a and the only Frame on the Page, meaning your 14-16-18 other Frames will be embedded/nested in that 1st Frame... Hum..., this is getting interesting, ah-ah...! :twisted:
I'm nearly surprised that iMacros doesn't get a bit lost with the Frame Counter when you have Frames nested into Frames like it seems in your Case... But hum..., bad Design of the Site already I would think... :wink:

But OK, for attaching a "workable" Full HTML Saveas of your Page, you need to place/copy the Main 'BPO _ flexmls Web.htm' Page into some empty/Temp Folder, together with the 'BPO _ flexmls Web_files' Folder and to zip the whole Temp Folder...
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all 3 Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
chivracq
 
Posts: 6294
Joined: Sat Apr 13, 2013 6:07 am
Location: Amsterdam (NL)

Re: Dynamic Frame Tracking

by azbob on Wed Jun 14, 2017 12:46 am

It works!!!
You nailed it with "Yep, all this part is about the Var 'Frame_nb' not being defined in the '.iim' Macro...
Nice that you define a Var with the same name in your '.js', but the '.iim' Macro doesn't know anything about what happens in the '.js' part unless you declare a Var with the name that you use in the '.iim' Macro using 'iimSet()'..., which is still missing..." See you wor
Here is a correct version. I want to play with timeout now so it won't spend as long looking for anchor.
Code: Select all
iimPlay('++AATEST_PRE_LOOP.iim');
//declarations
var framenumber;
var retcode;
var Frame_Nb;
var fnb=(15);

var find_number =  "CODE:";
find_number += "SET !ERRORIGNORE YES" + "\n";
find_number += "SET !EXTRACT NULL" + "\n";
find_number +=  "FRAME F={{Frame_Nb}}" + "\n";
find_number +=  "TAG POS=1 TYPE=TD ATTR=TXT:High" + "\n";
find_number +=  "TAG POS=R1 TYPE=TD ATTR=ID:hiLP EXTRACT=TXT" + "\n";

for(; fnb <= 18; fnb++){
   //alert(fnb);
   iimSet("Frame_Nb",fnb);
   iimPlay(find_number);
   //retcode=iimPlay(find_number);
   var frame_valid = iimGetLastExtract(1);
   //alert(iimGetLastExtract(1));
   iimSet("framenumber",fnb);
   //alert(frame_valid);
   //var frame_valid = iimGetExtract();
   //iimSet("RetCode",retcode);
   //iimSet("extract",frame_valid);

//if extract was frame number than save frame number and play rest of macro
   if (frame_valid !== "#EANF#")
        {
        iimSet("Frame_Nb",fnb)
   var framenumber=fnb;
   break;
   }
   
   }
iimPlay('++AATEST_POST_LOOP.iim');

One of the files is to big even zipped..so if you still want it I can send to dropbox. Let me know
Attachments
BPO _ flexmls Web.zip
the smallest of the files
(1.77 KIB) Downloaded 33 times
azbob
 
Posts: 76
Joined: Mon Sep 21, 2009 11:16 am

Re: Dynamic Frame Tracking

by chivracq on Wed Jun 14, 2017 7:31 am

azbob wrote:It works!!!
You nailed it with "Yep, all this part is about the Var 'Frame_nb' not being defined in the '.iim' Macro...
Nice that you define a Var with the same name in your '.js', but the '.iim' Macro doesn't know anything about what happens in the '.js' part unless you declare a Var with the name that you use in the '.iim' Macro using 'iimSet()'..., which is still missing..." See you wor

Here is a correct version. I want to play with timeout now so it won't spend as long looking for anchor.
Code: Select all
iimPlay('++AATEST_PRE_LOOP.iim');
//declarations
var framenumber;
var retcode;
var Frame_Nb;
var fnb=(15);

var find_number =  "CODE:";
find_number += "SET !ERRORIGNORE YES" + "\n";
find_number += "SET !EXTRACT NULL" + "\n";
find_number +=  "FRAME F={{Frame_Nb}}" + "\n";
find_number +=  "TAG POS=1 TYPE=TD ATTR=TXT:High" + "\n";
find_number +=  "TAG POS=R1 TYPE=TD ATTR=ID:hiLP EXTRACT=TXT" + "\n";

for(; fnb <= 18; fnb++){
   //alert(fnb);
   iimSet("Frame_Nb",fnb);
   iimPlay(find_number);
   //retcode=iimPlay(find_number);
   var frame_valid = iimGetLastExtract(1);
   //alert(iimGetLastExtract(1));
   iimSet("framenumber",fnb);
   //alert(frame_valid);
   //var frame_valid = iimGetExtract();
   //iimSet("RetCode",retcode);
   //iimSet("extract",frame_valid);

//if extract was frame number than save frame number and play rest of macro
   if (frame_valid !== "#EANF#")
        {
        iimSet("Frame_Nb",fnb)
   var framenumber=fnb;
   break;
   }
   
   }
iimPlay('++AATEST_POST_LOOP.iim');

One of the files is to big even zipped..so if you still want it I can send to dropbox. Let me know

Well, good to hear, ah-ah...!
But, well..., I had already "nailed" it about the undefined 'Frame_nb' (that you now call 'Frame_Nb') in my 1st and 2nd Replies as well, ah-ah...!
And your "iimSet("Frame_Nb",fnb);" now takes care of that... Good...

>>>

Euh, yep, still interested by your Full HTML Saveas, the 2nd Attachment is the same like the first one...
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all 3 Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
chivracq
 
Posts: 6294
Joined: Sat Apr 13, 2013 6:07 am
Location: Amsterdam (NL)

Re: Dynamic Frame Tracking

by azbob on Wed Jun 14, 2017 6:05 pm

Here is link to files.

https://www.dropbox.com/sh/xjdshdwxrqpm ... K2yCa?dl=0

I hope I created them correctly.
Let me know if you need anything else.
Thanks,
Bob
azbob
 
Posts: 76
Joined: Mon Sep 21, 2009 11:16 am

Next

Return to Data Extraction and Web Screen Scraping

Who is online

Users browsing this forum: No registered users and 2 guests

-->