How to know until where I have reached in datasource file?

Support for iMacros. The iMacros software is the unique solution for automating every activity inside a web browser, for data extraction and web testing.

Moderators: Community Moderators, iMacros Moderators

Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the Google search box (at the top of each forum page) to see if a similar problem or question has already been addressed. This will search the entire contents of the forums as well as the iMacros Wiki.
3. We can respond much faster to your posts if you include the following information:

CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST

Answering your own posts (e.g. attempting to "bump" your topic) drops your topic from the list of unanswered threads, so it may actually receive less views.

How to know until where I have reached in datasource file?

by faizzsheikh on Mon Jan 15, 2018 1:38 am

Hello There,

I am using 'datasource' command which includes (CSV) a list of urls to perform an action. Can I get a log report showing which URL the macros has reached when it got closed, this is because the browser gets an error and closes abruptly. I tried SET !FILELOG but it doesnt include the URL.
faizzsheikh
 
Posts: 9
Joined: Thu Nov 28, 2013 12:26 am

Re: How to know until where I have reached in datasource fil

by chivracq on Mon Jan 15, 2018 2:19 pm

faizzsheikh wrote:Hello There,

I am using 'datasource' command which includes (CSV) a list of urls to perform an action. Can I get a log report showing which URL the macros has reached when it got closed, this is because the browser gets an error and closes abruptly. I tried SET !FILELOG but it doesnt include the URL.

Answer is YES and there are ([strike]probably[/strike]) several ways to achieve what you want... I can find 3-4-5 different ways if I "think" for 10 sec... :idea:

But same initial Reply like 2.5 years ago, which was the last time I replied to one of your Threads and tried to help you, I'm a bit "tired" of asking the same "things" each time... :roll:

And '!FILELOG' doesn't play any Role on the Content of the Log afaik, even if I've never used that Command as it is only supported on [iMB + IE] (I (mostly) only use the Free Add-on on FF), it only controls its Location...
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all 3 Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
chivracq
 
Posts: 6966
Joined: Sat Apr 13, 2013 6:07 am
Location: Amsterdam (NL)

Re: How to know until where I have reached in datasource fil

by faizzsheikh on Mon Jan 22, 2018 3:06 am

Extremely sorry about that! :) will make sure from next time to include the config.

iMacros Version: 10.0.1
Browser: Chrome Version 63.0.3239.132
OS: Windows 7, 32 bit

Please help me out, thank you.
faizzsheikh
 
Posts: 9
Joined: Thu Nov 28, 2013 12:26 am

Re: How to know until where I have reached in datasource fil

by chivracq on Mon Jan 22, 2018 4:39 am

faizzsheikh wrote:Extremely sorry about that! :) will make sure from next time to include the config.

Code: Select all
iMacros Version: 10.0.1
Browser: Chrome Version 63.0.3239.132
OS: Windows 7, 32 bit


Please help me out, thank you.

OK, good, we finally have your FCI... Be aware I will probably not even react to your (next) Thread(s) if you don't mention this Info when you open a Thread, like I ignored some previous Thread you had opened previously...

But..., oh...!, v10.0.1 for CR, OK..., so you are not even on iMB or IE, then '!FILELOG' is not even supported on CR..., unless it's been implemented in the v10.0.1 for CR Version that got recently released, but that's not mentioned in the Wiki...

Hum..., I mentioned I had 5 Sols, let's see if I can find them back, it's the morning for me now, not completely awake yet, I'm usually sharper and more "Creative" in the evening, ah-ah...! :oops:

1- One "obvious" possible Solution, a bit like the 'Log File' Functionality, would be to enable the 'Profiler' to check if the URL of an 'URL GOTO' Command gets recorded.
And yep it does...!, from checking on some older 'Firefox-Profiler_xxx.xml' File of mine I can find back in my iMacros 'Downloads' Folder, run on some "antic" v7.30 for FF Script, probably using v8.8.2 for FF. I reckon it will still work the same in later Versions and on CR.

But hum..., I'm not a big-big Fan of the Profiler to be honest, even if it can be very useful sometimes for Debug Purpose or for monitoring the Load Time for Web-Pages/Sites, but it slows down the Speed of Macros by about 30% in my Experience, and that was a bit of a Pb in my Case and for the kind of Macros that I run...

2- The next Solution(s) involve(s) some (custom) Logging, which is actually a Good Practice anyway, I find myself. I find it more flexible than the Profiler as you decide yourself what parts of your Scripts that might go wrong you log to some '.CSV' or '.TXT' (or even '.LOG') File using the 'SAVEAS' Mechanism. You can even implement some "Double" Logging like I do myself, with one "big" long-term Log File each time you run that same Macro and one mini-Temp Log File corresponding to the current Run and that gets deleted each time you are going to run the same Macro again, or at Loop=1 only if you loop your Macro. And that Temp File is easier to check if your Script aborted for any Reason.

3- If your Log File has '.TXT' or '.LOG' Extension, you can even open it (automatically) directly from your Browser (from a Bookmark for example), or even at the end of your Script if it's still running, or at the end of each Loop to update it in real time, in some 2nd Tab.

4- That same Logging Info can be kept as well in your OS Clipboard to be pasted in some Temp '.TXT' File at any moment if you have any doubts about how your Script finished, about the current/last Loop and last URL for example.

5- And "surfing" on '4-', you can automatically display that Info in some Conditional 'PROMPT' or the iMacros Side Panel (using 'EVAL()' + 'MacroError()') at the end of your Script if it finished "properly" as some kind of mini-Report with the Info that you want. If you don't get the final Report, then you know that stg went wrong and you can then check the Content of the Clipboard and/or your Log File.

Hum, mini-Side-Note, I find it not a Best Practice to rely on the Clipboard as it can be a bit dangerous, if you are doing other "Things" while your Script is running and you want to use the Clipboard manually, ah-ah...!

OK, good luck and mention for what Solution(s) you go and maybe post your final Script if that can help other Users...
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all 3 Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
chivracq
 
Posts: 6966
Joined: Sat Apr 13, 2013 6:07 am
Location: Amsterdam (NL)

Re: How to know until where I have reached in datasource fil

by faizzsheikh on Mon Jan 22, 2018 8:11 am

Thanks for your help, I have used the SAVEAS command to generate a log file that indicates the last URL crawled from the DATASOURCE file. I also found that the command ADD !EXTRACT {{!URLCURRENT}} can also be very helpful to indicate the last parsed url or event (Know more on this thread: viewtopic.php?f=7&t=5330). Below is how my final script looks like.

About the imacros version I got it from the description mentioned in the chrome extension page: https://chrome.google.com/webstore/deta ... lemp?hl=en it says iMacros for Chrome 10.0.1

Code: Select all
VERSION BUILD=11.5.499.3066
TAB T=1
TAB CLOSEALLOTHERS
SET !WAITPAGECOMPLETE YES
SET !ERRORIGNORE YES
SET !TIMEOUT_PAGE 2
TAB CLOSEALLOTHERS
SET !PLAYBACKDELAY 0.2
SET !DATASOURCE D:\imacros\mkt-bot\scoopit\datasource.csv
SET !LOOP 2
SET !DATASOURCE_LINE {{!LOOP}}


'Add URL to the field
TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:urlToResolve CONTENT={{!COL3}}
WAIT SECONDS=2

'Click url submit button provided
TAG POS=1 TYPE=DIV ATTR=ID:urlChooserButton
WAIT SECONDS=3

'Enter the description
EVENT TYPE=CLICK SELECTOR="#h_scoopitWindowPopup>DIV>DIV:nth-of-type(2)>DIV>DIV>DIV:nth-of-type(3)>DIV>DIV>DIV:nth-of-type(2)>DIV" BUTTON=0
EVENTS TYPE=KEYPRESS SELECTOR="#h_scoopitWindowPopup>DIV>DIV:nth-of-type(2)>DIV>DIV>DIV:nth-of-type(3)>DIV>DIV>DIV:nth-of-type(2)>DIV" CHARS={{!COL2}}
WAIT SECONDS=1

'Enter the tag, only one tag can be added, multiple tags needs to be added manually
TAG POS=1 TYPE=INPUT:TEXT ATTR=CLASS:tagsEditorInput<SP>tagsEditor-input<SP>ui-autocomplete-input CONTENT={{!COL4}}
WAIT SECONDS=1

'Click the publish button and done
TAG POS=3 TYPE=DIV ATTR=TXT:Publish

'Log indicator
ADD !EXTRACT {{!COL3}}


SAVEAS TYPE=EXTRACT FOLDER=D:\imacros\mkt-bot\scoopit FILE=test.TXT
faizzsheikh
 
Posts: 9
Joined: Thu Nov 28, 2013 12:26 am

Re: How to know until where I have reached in datasource fil

by chivracq on Mon Jan 22, 2018 8:36 am

faizzsheikh wrote:Thanks for your help, I have used the SAVEAS command to generate a log file that indicates the last URL crawled from the DATASOURCE file. I also found that the command ADD !EXTRACT {{!URLCURRENT}} can also be very helpful to indicate the last parsed url or event (Know more on this thread: viewtopic.php?f=7&t=5330). Below is how my final script looks like.

About the imacros version I got it from the description mentioned in the chrome extension page: https://chrome.google.com/webstore/deta ... lemp?hl=en it says iMacros for Chrome 10.0.1

Code: Select all
VERSION BUILD=11.5.499.3066
TAB T=1
TAB CLOSEALLOTHERS
SET !WAITPAGECOMPLETE YES
SET !ERRORIGNORE YES
SET !TIMEOUT_PAGE 2
TAB CLOSEALLOTHERS
SET !PLAYBACKDELAY 0.2
SET !DATASOURCE D:\imacros\mkt-bot\scoopit\datasource.csv
SET !LOOP 2
SET !DATASOURCE_LINE {{!LOOP}}

'Add URL to the field
TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:urlToResolve CONTENT={{!COL3}}
WAIT SECONDS=2

'Click url submit button provided
TAG POS=1 TYPE=DIV ATTR=ID:urlChooserButton
WAIT SECONDS=3

'Enter the description
EVENT TYPE=CLICK SELECTOR="#h_scoopitWindowPopup>DIV>DIV:nth-of-type(2)>DIV>DIV>DIV:nth-of-type(3)>DIV>DIV>DIV:nth-of-type(2)>DIV" BUTTON=0
EVENTS TYPE=KEYPRESS SELECTOR="#h_scoopitWindowPopup>DIV>DIV:nth-of-type(2)>DIV>DIV>DIV:nth-of-type(3)>DIV>DIV>DIV:nth-of-type(2)>DIV" CHARS={{!COL2}}
WAIT SECONDS=1

'Enter the tag, only one tag can be added, multiple tags needs to be added manually
TAG POS=1 TYPE=INPUT:TEXT ATTR=CLASS:tagsEditorInput<SP>tagsEditor-input<SP>ui-autocomplete-input CONTENT={{!COL4}}
WAIT SECONDS=1

'Click the publish button and done
TAG POS=3 TYPE=DIV ATTR=TXT:Publish

'Log indicator
ADD !EXTRACT {{!COL3}}

SAVEAS TYPE=EXTRACT FOLDER=D:\imacros\mkt-bot\scoopit FILE=test.TXT

Yep, very good, more or less what I meant in my previous Reply, even if you let it to some very "basic" Logging, ah-ah...!, but OK if that meets your Needs... :D

Be careful with '!URLCURRENT' as it was not working properly in v8.8.4 for CR, and the Version History for v10.0.1 (finally updated...!) doesn't mention anything about any Change for this Command...
Hum..., and I see that '!PLAYBACKDELAY' is now supported on v10.0.1 for CR, Okay..., good to know...
But '!WAITPAGECOMPLETE' that I see you use in your Script is still not supported on CR, according to the Wiki for that Command, and it is not mentioned either in the Version History... :?
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all 3 Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
chivracq
 
Posts: 6966
Joined: Sat Apr 13, 2013 6:07 am
Location: Amsterdam (NL)


Return to General Support & Discussions

Who is online

Users browsing this forum: No registered users and 5 guests

-->