Page (feed) wont load fully (Pale Moon Questions)

Discussions and Tech Support specific to the iMacros Firefox add-on.
Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information: CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
Post Reply
Lightja
Posts: 20
Joined: Tue Aug 30, 2016 3:40 am

Page (feed) wont load fully (Pale Moon Questions)

Post by Lightja » Wed Nov 02, 2016 12:41 am

I'm trying to extract a list of followers for users on pinterest.

Take this pinterest for example: https://www.pinterest.com/olympicfinishes/followers/

When you go to that page, the first ~80 followers will load. When you scroll down, more begin to load. I'm trying to extract the full list of followers, in this case 4105 of them. The problem is that almost every time the page stops loading after the first 300-2000 followers. The first 10 people I did this for worked no problem, one of which had 15000 followers, and it loaded them all no problem. Now for some reason I can't get past the first 1000 or so.

Is there any way to make sure the page loads? Is there any way to tell in the middle of a macro that a command didn't work? I'm using javascript so iimPlayCode(macro) will return a 1 if everything goes well and 0 if something fails, but it wont return until every command has failed. So if it fails at the 300th follower, it still has to run for 3800 more, which for a 6 second fail time each takes 6 hours to finish to tell me it didn't work. Even if I cut it down to 3 second fail (can't go much lower than that), it still takes 3 hours.

Pale Moon users, are there any things I can do to increase the speed / loading reliability of Pale Moon in general? It doesn't need to be foolproof, it just needs to fully work more than 5% of the time, which is what it's currently working at.

Pale Moon 26.4.1
Imacros 8.8.7.1-signed
Windows 10 Pro
Latest System Information:
iMacros 11.1
Windows 10 Enterprise
Firefox v48.0.2 - iMacros for Firefox v8.9.7
Chrome v53.0.2785.89 - iMacros for Chrome v8.4.4
chivracq
Posts: 9309
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Page (feed) wont load fully (Pale Moon Questions)

Post by chivracq » Wed Nov 02, 2016 2:14 pm

Lightja wrote:I'm trying to extract a list of followers for users on pinterest.

Take this pinterest for example: https://www.pinterest.com/olympicfinishes/followers/

When you go to that page, the first ~80 followers will load. When you scroll down, more begin to load. I'm trying to extract the full list of followers, in this case 4105 of them. The problem is that almost every time the page stops loading after the first 300-2000 followers. The first 10 people I did this for worked no problem, one of which had 15000 followers, and it loaded them all no problem. Now for some reason I can't get past the first 1000 or so.

Is there any way to make sure the page loads? Is there any way to tell in the middle of a macro that a command didn't work? I'm using javascript so iimPlayCode(macro) will return a 1 if everything goes well and 0 if something fails, but it wont return until every command has failed. So if it fails at the 300th follower, it still has to run for 3800 more, which for a 6 second fail time each takes 6 hours to finish to tell me it didn't work. Even if I cut it down to 3 second fail (can't go much lower than that), it still takes 3 hours.

Pale Moon users, are there any things I can do to increase the speed / loading reliability of Pale Moon in general? It doesn't need to be foolproof, it just needs to fully work more than 5% of the time, which is what it's currently working at.

Code: Select all

Pale Moon 26.4.1
Imacros 8.8.7.1-signed
Windows 10 Pro
Latest System Information:

Code: Select all

iMacros 11.1
Windows 10 Enterprise
Firefox v48.0.2 - iMacros for Firefox v8.9.7
Chrome v53.0.2785.89 - iMacros for Chrome v8.4.4
I use Pale Moon as well (iMacros for FF v8.8.2, PM v26.3.3, Win10-x64) which I consider much more reliable and performing than pure FF, interesting btw to see that you are using v8.8.7 which seems to work on PM v26.4.1.

For your Qt, yep, there are several Techniques, but I can't have a look at this Site which absolutely wants me to register and you didn't post your Script so I don't know if you loop your Script for each Follower Extraction + a mini Scroll, or if you extract say 10 Followers + a "big" Scroll, but to "speed up" the Execution, if you extract per 10, you could use '!TIMEOUT-STEP=3' for say the first 2 Extractions and lower it to 1 or 0 for the next 8 Followers.
For Reliability, you can always add a "fake" TAG+EXTRACT+EVAL to first check if the next Follower is available, in order to add a Conditional 'WAIT' or conditionally increase '!TIMEOUT_STEP' or add a Conditional Extra Scroll-down before doing the "real" Extract.
And even when you finish a Loop, you can already check if there is/are already 1-2-3 next Followers available on the Page to return that Info to your Main '.js' Script to conditionally act on that Info... => for a Conditional Scroll, or Reload, or Abort, or Prompt, or Logging, whatever...!
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE').
- I don't even read the Qt if that (required) Info is not mentioned...!
- Script & URL help a lot for more "educated" Help...
Post Reply