Resume extraction though Firefox browser crash

Discussions and Tech Support related to website data extraction, screen scraping and data mining using iMacros.

Moderators: Community Moderators, iMacros Moderators

Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the Google search box (at the top of each forum page) to see if a similar problem or question has already been addressed. This will search the entire contents of the forums as well as the iMacros Wiki.
3. We can respond much faster to your posts if you include the following information:

CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST

Answering your own posts (e.g. attempting to "bump" your topic) drops your topic from the list of unanswered threads, so it may actually receive less views.

Resume extraction though Firefox browser crash

by Jolene on Tue Jun 06, 2017 1:45 am

Hi everyone, i'm facing issue with Firefox browser. Once the browser crashes , I have to start all over with my extraction.

Is there a way for me to continue extracting from where it stopped rather than starting all over again from the beginning using JavaScript?
Jolene
 
Posts: 2
Joined: Tue Jun 06, 2017 1:06 am

Re: Resume extraction though Firefox browser crash

by chivracq on Tue Jun 06, 2017 2:48 am

Jolene wrote:Hi everyone, i'm facing issue with Firefox browser. Once the browser crashes , I have to start all over with my extraction.

Is there a way for me to continue extracting from where it stopped rather than starting all over again from the beginning using JavaScript?

FCIM...! :mrgreen: (Read my Sig...)

Your Post is very vague...! :roll:
=> Generic Answer...: It depends on your Site, + on your Script, + if you loop your Script, + if your Script runs 24/7, etc..., we can't really tell if you don't post your Script to see what it is doing exactly...
But yep, there are "Techniques" by adding some Logging to your Script for keeping track of "how far" your Script went until "stg" happened and your Script stopped unexpectedly..., or to restart FF and/or your Script automatically before some "statistically average" period of time before you can expect that FF will crash...

And maybe you should first try to solve the Crashes, that's not really normal..., by maybe dedicating a mini FF-Profile to your Extraction with only the few necessary Add-ons added to that Profile... Or you can use Pale Moon as well which doesn't crash from Memory Leaks like FF does...
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
chivracq
 
Posts: 5730
Joined: Sat Apr 13, 2013 6:07 am
Location: Amsterdam (NL)

Re: Resume extraction though Firefox browser crash

by Jolene on Tue Jun 06, 2017 7:11 pm

Thanks for the reply.
As i'm working on a confidential project, I won't be able to share my code in this forum.
Hope you can understand.

As for FF crash i will look into it to resolve it. Meanwhile i have another problem which i guess is about the same problem.

As iMacros add-on have a start and stop button located at the left side. Let's treat it as the browser doesn't crash, everything runs smoothly and i wanna it to stop the extraction for today(Day 1) and resume extraction on the next working day(Day 2).

Scenario(How i want the program to work):
(Day 1) Do my extraction for the whole day, stop extracting at the end of the day.
(Day 2) Continue with extraction BUT starting from where i left off on (Day 1).

Problem Faced:
It will start extracting from scratch regardless, rather than starting from where i left off on (Day 1).

Question:
In what way can i approach JavaScript to ensure that the extraction will not start from scratch?

With suggestion given previously by chivracq on "adding some Logging to your Script for keeping track of "how far" your Script"
Could you kindly explain what is "Logging" and how i could apply it? Maybe some example will be appreciated.
Jolene
 
Posts: 2
Joined: Tue Jun 06, 2017 1:06 am

Re: Resume extraction though Firefox browser crash

by chivracq on Tue Jun 06, 2017 7:35 pm

Jolene wrote:Thanks for the reply.
As i'm working on a confidential project, I won't be able to share my code in this forum.
Hope you can understand.

As for FF crash i will look into it to resolve it. Meanwhile i have another problem which i guess is about the same problem.

As iMacros add-on have a start and stop button located at the left side. Let's treat it as the browser doesn't crash, everything runs smoothly and i wanna it to stop the extraction for today(Day 1) and resume extraction on the next working day(Day 2).

Scenario(How i want the program to work):
(Day 1) Do my extraction for the whole day, stop extracting at the end of the day.
(Day 2) Continue with extraction BUT starting from where i left off on (Day 1).

Problem Faced:
It will start extracting from scratch regardless, rather than starting from where i left off on (Day 1).

Question:
In what way can i approach JavaScript to ensure that the extraction will not start from scratch?

With suggestion given previously by chivracq on "adding some Logging to your Script for keeping track of "how far" your Script"
Could you kindly explain what is "Logging" and how i could apply it? Maybe some example will be appreciated.

Oh, good that you follow up, but sorry 'FCIM" again from my Side for me to read your Reply, I normally don't even read Opening Posts when FCI is missing and only rarely reply for first time Posters, which is not your case anymore, ah-ah...!, I am afraid you missed the first part of my Reply... :idea:
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
chivracq
 
Posts: 5730
Joined: Sat Apr 13, 2013 6:07 am
Location: Amsterdam (NL)


Return to Data Extraction and Web Screen Scraping

Who is online

Users browsing this forum: Bing [Bot] and 2 guests

Website Monitoring