Problems with copying content of page

Discussions and Tech Support related to website data extraction, screen scraping and data mining using iMacros.
Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information: CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
chivracq
Posts: 9515
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Problems with copying content of page

Post by chivracq » Sun Nov 24, 2019 9:24 pm

brokenzeus wrote:
Sun Nov 24, 2019 7:39 pm
Thanks for all the help. Appreciate it. I took the uploaded page offline. As far as the extension goes i tried saving the page on a windows computer and there it saved as .json.

I tried your script on my mac in chrome but it's still not working. It does not copy the text from page 2 into the text box on page1. I tried it on Win 10 on FF 70.0.1 (64-bits) but still not working. When i tried to run the script you said would only work on FF i get an error that EVENTS is not supported.

But i do not understand why it is not working. You say that it works on your browser and computer but on mine it's not. Very strange.

And yes in Leuven most people talk Dutch instead of French because Leuven is part of the Flemish speaking region. It's quite tricky.

Ha-han...!?, grrrr....!, bummer...!, I really thought "Case solved, and one (more) Happy Customer, youpidoo...!!", ah-ah...! :P :( :cry: :evil:

But one important Thing (again)...: Did you check in CR (all OS'es) that you have the "Allow access to URLs" Setting in the iMacros Options enabled...?

Then hum, OK..., it could be a Timeout Issue...
Because when I test I already have 'Page2' fully loaded as a Local File in 'TAB_2', but for you that Content needs to be generated from the Server, and 155Kb could take quite a few Seconds, like 5 Sec at a 30Kbs Connection, even if you probably have a much quicker Connection, but because that Page/Tab gets opened from the Page on 'TAB_1', it's possible that iMacros (still on 'TAB_1') is not really aware of what is happening on 'TAB_2' and doesn't "wait" for 'TAB_2' and its Content to (fully) load...
=> Try adding a "WAIT SECONDS=5" when you switch to "TAB T=2", just before the 'EXTRACT', hum OK, try this one with a bit of Debugging...:

Code: Select all

VERSION BUILD=8820413 RECORDER=FX
SET !TIMEOUT_PAGE 30
SET !TIMEOUT_STEP 5
SET !EXTRACT_TEST_POPUP NO

TAB T=1
URL GOTO=https://kultoical.dries007.net/
TAG POS=1 TYPE=INPUT:DATE ATTR=ID:start CONTENT=2019-11-01
TAG POS=1 TYPE=INPUT:DATE ATTR=ID:end CONTENT=2020-01-31
TAG POS=1 TYPE=BUTTON ATTR=TXT:Next
TAG POS=1 TYPE=A ATTR=TXT:https://webwsp.aps.kuleuven.be/sap/opu/odata/sap/zc_ep_sched*
WAIT SECONDS=1

TAB T=2
WAIT SECONDS=5
'URL GOTO=file:///D:/TEMP/iMacros/Temp/_Forum%20Cases/brokenzeus/Page2/Page2
'Recorded:
'TAG POS=1 TYPE=PRE ATTR=TXT:{"d":{"results":[{"__metadata":{"id":"https://WEBWSP.APS.KUL*
'>
TAG POS=1 TYPE=PRE ATTR=TXT:* EXTRACT=TXT
WAIT SECONDS=1

TAB T=1
SET !VAR1 EVAL("var s='{{!EXTRACT}}'; var z; z=s.substr(0,1000); z;")
'PROMPT EXTRACT:<BR><BR>_{{!EXTRACT}}_
PROMPT EXTRACT:<BR><BR>_{{!VAR1}}_
WAIT SECONDS=1
TAG POS=1 TYPE=TEXTAREA ATTR=ID:json CONTENT={{!EXTRACT}}
The Content of '!EXTRACT' in the 'PROMPT' might get truncated..., hum, and yep it is indeed, but the 'PROMPT' Command in these v10.0.x Versions for CR/FF is completely Buggy, ah-ah...! :shock: , so OK, I use a Temp Var ('!VAR1') where I deliberately truncate '!EXTRACT' to only the first 1000 Chars, or the Script will hang upon clicking on the 'Close' button as the '<OK>' Button is not reachable otherwise...

Stg you could try also would be to test in the same Conditions like I did, by first saving locally that 'Page2' to your Mac, then drag it using the MacOS File Explorer ('File Finder' I think...?) to some already opened 'TAB_2' where the File should get automatically displayed... (But you'll need to (temporarily) disable the Click on the Link, as you don't want it to open the 'TAB_2' which is already opened and loaded with Content...)

And I could try myself to update my CR76 to CR78 in case CR78 is "breaking" it, but hum..., I have a very limited/metered Int. Co., and I'll need to switch to some other Co. (which doesn't work at the moment) to be able to download the 30/40/60Mb(?) needed to update that stupid Browser...
Same with FF Quantum that I haven't used in maybe 1 year and that still must be at FF62 or maybe FF65 and that will also want to update itself...

>>>
I tried it on Win 10 on FF 70.0.1 (64-bits) but still not working. When i tried to run the script you said would only work on FF i get an error that EVENTS is not supported.

Yep, normal, only iMacros for FF v10.0.2 ('Free'/'PE') works on FF Quantum (v[57-70(+)]) where the 'EVENT' Mode is not supported (anymore/yet), Reason why I mentioned "v8.9.7 for FF + FF v55.0.3"...
Last edited by chivracq on Tue Nov 26, 2019 7:41 am, edited 1 time in total.
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE').
- I don't even read the Qt if that (required) Info is not mentioned...!
- Script & URL help a lot for more "educated" Help...
brokenzeus
Posts: 9
Joined: Sat Nov 23, 2019 5:29 pm

Re: Problems with copying content of page

Post by brokenzeus » Tue Nov 26, 2019 6:49 am

Was quite busy yesterday but i was checking if I enabled the option in chrome and I did. But then you said it could maybe be a timeout issue and in the chrome settings the playback speed was set to fast. I changed it to medium and boom it works. So you were right that it's a timeout issue. The script works without a problem. I didn't add the wait 5s but i changed playback speed and that seemed to fix it.

Thanks for all your help appreciate it. :D :D

Now a small side question: Do you think it would be possible to run this macros on a smartphone? Like for example using shortcuts on IOS? It does support javascript but i don't think it supports iMacros.
chivracq
Posts: 9515
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Problems with copying content of page

Post by chivracq » Tue Nov 26, 2019 8:42 am

brokenzeus wrote:
Tue Nov 26, 2019 6:49 am
Was quite busy yesterday but i was checking if I enabled the option in chrome and I did. But then you said it could maybe be a timeout issue and in the chrome settings the playback speed was set to fast. I changed it to medium and boom it works. So you were right that it's a timeout issue. The script works without a problem. I didn't add the wait 5s but i changed playback speed and that seemed to fix it.

Thanks for all your help appreciate it. :D :D

Now a small side question: Do you think it would be possible to run this macros on a smartphone? Like for example using shortcuts on IOS? It does support javascript but i don't think it supports iMacros.

Oh, woaw...!, good to hear about the Timeout/Speed, 'Playback Speed' set to "Fast" is "normal" and "standard", I have never-NEVER-NEVER-ever-ever-EVER-EVER had to slow it down in the 10-12 years I have been using iMacros... :P

Oh...!, but I guess, as you are using v10.0.5 for CR, that you are probably using the '!PLAYBACKDELAY' Command which accepts not only "Full Seconds" (unlike '!REPLAYSPEED'), but "0.2" for example, which hum... alright..., is probably OK, but all Statements are "impacted' while probably only 1 or 2 Lines in your Script need that Delay, and I had deliberately chosen for "long" 'WAIT' Statements in my "last" Test Script to check if it would work..., for you then to shorten all those 'WAIT' Statements until you found the "tuned workable" Limit, ah-ah...! :wink:

>>>
Now a small side question: Do you think it would be possible to run this macros on a smartphone? Like for example using shortcuts on IOS? It does support javascript but i don't think it supports iMacros.

Well, on any SmartPhone, you'll need to be able to install iMacros in your Browser on that SmartPhone, be it CR or FF... :|

You already have a few Threads "about that" on the Forum, none "really" with a clear Solution, I think... :|

I have several "Ideas" about it, but I never (really) did any "real" Testing about that myself (because I didn't have a SmartPhone, ah-ah...! :P , and now yep, but only since pretty recently, and a seemingly pretty "old" one, Kado from a Friend... :D (Android v4.1.2 / Kernel v3.0.31: Samsung Galaxy S2 :P ), so I've in all "those years" "developed" other Solutions, or now simply use 'TeamViewer' if needed on that SmartPhone, but my "other Solutions" are pretty "robust", so I actually never need it, ah-ah...! :P

One of if not THE most "important" Reason(s) I'm "pretty" Advanced with Web-Automation is that I actually have very little/limited Access to Internet, (all 10 years, with some "easier" Periods from time to time..., now is one, even if it's a bit "costly"), and I never know "when" it will work (again), and for how long..., so my Scripts all know when to run..., and to wait... until Co. is back, and are all optimized to run as quick as possible, and to re-run if "anything" went wrong, and I still have a "40-60 Hours "Presence" on Internet" on Average daily. (+ Mod-"Duty" and Help on the Forum, that I do (mostly) "manually"...)

But I would say, about iMacros on a SmartPhone, if my short Answer didn't completely answer your Qt, => open a New Thread about that..., (in the "correct" Sub-Forum => 'General'), and find (and link to) the few already existing and relevant Threads..., and I'll elaborate further... :wink:
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE').
- I don't even read the Qt if that (required) Info is not mentioned...!
- Script & URL help a lot for more "educated" Help...
Post Reply