exclude content of a txt/csv file when running script

Discussions and Tech Support specific to the iMacros Firefox add-on.
Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information: CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
disguised sensei
Posts: 32
Joined: Thu Sep 29, 2016 9:44 am

exclude content of a txt/csv file when running script

Post by disguised sensei » Thu Sep 29, 2016 10:36 am

iMacros for Firefox version: 9030808
S.O.: Windows 10 64 bit
Browser: Firefox 49.0.1

hello community! I need your valuable knowledge. Please bear with a beginner. You are going to laugh at my script----- but i need to make it work :)

I want to do this: At redhat there are a bunch of websites I'd like to open in new tabs in firefox. Here is the url: https://www.redhat.com/archives/libvir- ... /date.html Please have a look to understand. This site contains many urls. But I have gone through some already and some I simply don't need. Therefore I made a list with urls which I don't need to open. I called it URLs.txt. and it's in the Datasources folder. So it's a text file and not a .csv file. (Is that a problem?)

Then I have created 3 iim-files:

1. excludedURLs.iim which looks like this:

Code: Select all

VERSION BUILD=9030808 RECORDER=FX
TAB T=1
SET !DATASOURCE URLs.txt
2. doNothing.iim which simply has this content:

Code: Select all

VERSION BUILD=9030808 RECORDER=FX
3. openNewTabWithURL.iim which has this content:

Code: Select all

VERSION BUILD=9030808 RECORDER=FX
TAB T=1
EVENT TYPE=CLICK SELECTOR="HTML>BODY>DIV:nth-of-type(2)>DIV:nth-of-type(6)>DIV:nth-of-type(2)>DIV>DIV>DIV>DIV>DIV>A:nth-of-type(2)" BUTTON=0 MODIFIERS="ctrl"
EVENT TYPE=CLICK SELECTOR="HTML>BODY>DIV:nth-of-type(2)>DIV:nth-of-type(6)>DIV:nth-of-type(2)>DIV>DIV>DIV>DIV>DIV:nth-of-type(2)>A:nth-of-type(2)" BUTTON=0 MODIFIERS="ctrl"
EVENT TYPE=CLICK SELECTOR="HTML>BODY>DIV:nth-of-type(2)>DIV:nth-of-type(6)>DIV:nth-of-type(2)>DIV>DIV>DIV>DIV>DIV:nth-of-type(3)>A:nth-of-type(2)" BUTTON=0 MODIFIERS="ctrl"
EVENT TYPE=CLICK SELECTOR="HTML>BODY>DIV:nth-of-type(2)>DIV:nth-of-type(6)>DIV:nth-of-type(2)>DIV>DIV>DIV>DIV>DIV:nth-of-type(4)>A:nth-of-type(2)" BUTTON=0 MODIFIERS="ctrl"
(This iim goes to 100, but I only show the first few. I am sure there is an easier way, but I didn't figure it out yet.)

Then I combined all these macros in this javascript file:

Code: Select all

var ret;

for (var i = 1; i <= 1; i++) {

   // 1) search for the urls in URLs.txt
   ret = iimPlay("exludedURLs.iim");

   // 2)if it finds url: play doNothing.iim
   if (ret == 1) {   // Url was found
      ret = iimPlay("doNothing.iim");
   }
   // 3)if it doesn't find the url: play openNewTabWithURL.iim  
   else if (ret != -101) {
      
      ret = iimPlay("openNewTabWithURL.iim");  

   }
  
   if (ret == -101) {   // User clicked the Stop button
      break;
   }
}
I found parts of this JavaScript code in this forum and changed it to my purpose, but it is not working. I don't know how to make it search and just exclude those numbers AND still run the code for the other urls. There is something missing in my code. I just don't know what to add and how to implement it. So mostly I need your expertise with the last javascript. Please help :)
Last edited by disguised sensei on Thu Sep 29, 2016 5:13 pm, edited 2 times in total.
chivracq
Posts: 9179
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: exclude content of a txt/csv file when running script

Post by chivracq » Thu Sep 29, 2016 1:09 pm

disguised sensei wrote:

Code: Select all

iMacros for Firefox version: 9030808
S.O.: Windows 10 64 bit
Browser: Firefox 49.0.1
hello community! I need your valuable knowledge. Please bear with a beginner. You are going to laugh at my script----- but i need to make it work :)

I want to do this: At redhat there are a bunch of websites I'd like to open in new tabs in firefox. Here is the url: https://www.redhat.com/archives/libvir- ... /date.html Please have a look to understand. This site contains many urls. But I have gone through some already and some I simply don't need. Therefore I made a list with urls which I don't need to open. I called it URLs.txt. and it's in the Datasources folder. So it's a text file and not a .csv file. (Is that a problem?)

Then I have created 3 iim-files:

1. exludedURLs.iim which looks like this:

Code: Select all

VERSION BUILD=9030808 RECORDER=FX
TAB T=1
SET !DATASOURCE URLs.txt
2. doNothing.iim which simply has this content:

Code: Select all

VERSION BUILD=9030808 RECORDER=FX
3. openNewTabWithURL.iim which has this content:

Code: Select all

VERSION BUILD=9030808 RECORDER=FX
TAB T=1
EVENT TYPE=CLICK SELECTOR="HTML>BODY>DIV:nth-of-type(2)>DIV:nth-of-type(6)>DIV:nth-of-type(2)>DIV>DIV>DIV>DIV>DIV>A:nth-of-type(2)" BUTTON=0 MODIFIERS="ctrl"
EVENT TYPE=CLICK SELECTOR="HTML>BODY>DIV:nth-of-type(2)>DIV:nth-of-type(6)>DIV:nth-of-type(2)>DIV>DIV>DIV>DIV>DIV:nth-of-type(2)>A:nth-of-type(2)" BUTTON=0 MODIFIERS="ctrl"
EVENT TYPE=CLICK SELECTOR="HTML>BODY>DIV:nth-of-type(2)>DIV:nth-of-type(6)>DIV:nth-of-type(2)>DIV>DIV>DIV>DIV>DIV:nth-of-type(3)>A:nth-of-type(2)" BUTTON=0 MODIFIERS="ctrl"
EVENT TYPE=CLICK SELECTOR="HTML>BODY>DIV:nth-of-type(2)>DIV:nth-of-type(6)>DIV:nth-of-type(2)>DIV>DIV>DIV>DIV>DIV:nth-of-type(4)>A:nth-of-type(2)" BUTTON=0 MODIFIERS="ctrl"
(This iim goes to 100, but I only show the first few. I am sure there is an easier way, but I didn't figure it out yet.)

Then I combined all these macros in this javascript file:

Code: Select all

var ret;

for (var i = 1; i <= 1; i++) {

   // 1) search for the url in URLs.txt
   ret = iimPlay("exludedURLs.iim");

   // 2)if it finds the word: play doNothing.iim
   if (ret == 1) {   // Url was found
      ret = iimPlay("doNothing.iim");
   }
   // 3)if it doesn't find the url: play openNewTabWithURL.iim  
   else if (ret != -101) {
      
      ret = iimPlay("openNewTabWithURL.iim");  

   }
  
   if (ret == -101) {   // User clicked the Stop button
      break;
   }
}
I found parts of this JavaScript code in this forum and changed it to my purpose, but it is not working. I don't know how to make it search and just exlude those numbers AND still run the code for the other urls. There is something missing in my code. I just don't know what to add and how to implement it. So mostly I need your expertise with the last javascript. Please help :)
Yep, more or less workable Approach, but for each new URL, you will need your 'ExcludedURLs.iim' Macro to loop through all Excluded_URLs again and again, which is maybe workable if you have 3-10 URL's to exclude, even if the time it will take in total will be multiplied by your total number of URL's to check...

Much more efficient is to use a Method I have already published a few times on the Forum which consists of using your Excluded_URLs.txt' Datasource (with '.txt' Extension, not with '.csv'...!) to open from your Macro (once) in a second Tab in your Browser (or make sure when you start your Script that it is already loaded in this 2nd Tab) for your Macro to switch for each URL to this 2nd Tab and simply try to extract this URL in just one Statement. If the Extract returns "#EANF#", you then know the URL was not found...!
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE').
- I don't even read the Qt if that (required) Info is not mentioned...!
- Script & URL help a lot for more "educated" Help...
disguised sensei
Posts: 32
Joined: Thu Sep 29, 2016 9:44 am

Re: exclude content of a txt/csv file when running script

Post by disguised sensei » Thu Sep 29, 2016 1:34 pm

hi chivracq! Thank you for your reply!

I am wondering if this information makes my task easier: when you scroll with your mouse over the urls, you will notice a certain pattern that redhat uses for the urls. All urls end with msg + a unique 5-digit number. eg.: "msg01234.html"

I think the macro can "read" these numbers on the page. I would like my script to be able to go through this page - which will be opened in Tab T=1 (https://www.redhat.com/archives/libvir- ... /date.html) and compare these unique 5-digit numbers with my text file.

If they DO NOT exist in my text file, then it should open the url in the next tab. I am ok with 10 tabs each. (So TAB 2 - 11 are opened). Later I can manually add the ones I worked through to my excluded list (URLs.txt). And then run the script again. This works, but it just opens all tabs regardless of my excluded list.

Sorry, I know this changes my question a bit, but this should be easier then right?

(BUT I HAVE NO CLUE WHAT TO ADD) :(
disguised sensei
Posts: 32
Joined: Thu Sep 29, 2016 9:44 am

Re: exclude content of a txt/csv file when running script

Post by disguised sensei » Thu Sep 29, 2016 6:52 pm

@chivracq: I tried to find the thread you were talking about, but I couldn't find it. Can you refer me to the right one? If something similar has been done, I will try to adjust it to my needs. Right now I just don't know how to proceed from here on without help.
chivracq
Posts: 9179
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: exclude content of a txt/csv file when running script

Post by chivracq » Thu Sep 29, 2016 7:28 pm

disguised sensei wrote:@chivracq: I tried to find the thread you were talking about, but I couldn't find it. Can you refer me to the right one? If something similar has been done, I will try to adjust it to my needs. Right now I just don't know how to proceed from here on without help.
Yep, sorry, a bit busy this afternoon (my time), several Approaches are possible for your Scenario...

But concerning "the Thread", simply open your DataSource (with '.txt' Extension, it doesn't work with '.csv') in a separate Tab in your Browser and treat it as a "normal" Web-Page to do some Extraction with iMacros.
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE').
- I don't even read the Qt if that (required) Info is not mentioned...!
- Script & URL help a lot for more "educated" Help...
disguised sensei
Posts: 32
Joined: Thu Sep 29, 2016 9:44 am

Re: exclude content of a txt/csv file when running script

Post by disguised sensei » Thu Sep 29, 2016 10:15 pm

ah, now I understand. I have now my text file open in firefox tab 2 - as if it were a regular website.

And I used this code to extract the numbers:

Code: Select all

VERSION BUILD=9030808 RECORDER=FX
TAB T=2
WAIT SECONDS=1
SET !EXTRACT_TEST_POPUP NO
TAG POS=1 TYPE=HTML ATTR=* EXTRACT=TXT
but now what? I don't know how to integrate this into my script --- so my extraction is excluded --- and this is what I need.
Last edited by disguised sensei on Fri Sep 30, 2016 4:05 pm, edited 1 time in total.
disguised sensei
Posts: 32
Joined: Thu Sep 29, 2016 9:44 am

Re: exclude content of a txt/csv file when running script

Post by disguised sensei » Fri Sep 30, 2016 4:04 pm

Can someone please direct me to a similar thread here in the forum. Really need a bit of your help.
chivracq
Posts: 9179
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: exclude content of a txt/csv file when running script

Post by chivracq » Fri Sep 30, 2016 4:53 pm

disguised sensei wrote:ah, now I understand. I have now my text file open in firefox tab 2 - as if it were a regular website.

And I used this code to extract the numbers:

Code: Select all

VERSION BUILD=9030808 RECORDER=FX
TAB T=2
WAIT SECONDS=1
SET !EXTRACT_TEST_POPUP NO
TAG POS=1 TYPE=HTML ATTR=* EXTRACT=TXT
but now what? I don't know how to integrate this into my script --- so my extraction is excluded --- and this is what I need.
disguised sensei wrote:Can someone please direct me to a similar thread here in the forum. Really need a bit of your help.
Yep, sorry, I'm always a bit busy during WE's, usually starting from Thursday... (I'm a DJ...)

The Principle is fairly simple but I guess you don't really understand it, you simply extract each URL from your Redhat Page and you reuse it as Argument to try to tag some Element on your Exclude Tab_2, and "#EANF#" tells you if your URL was not found.
You don't even need any '.js' Script to do so..., but search the Forum for Example Scripts using 'iimGetExtract()' and/or 'iimGetLastExtract()'...
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE').
- I don't even read the Qt if that (required) Info is not mentioned...!
- Script & URL help a lot for more "educated" Help...
disguised sensei
Posts: 32
Joined: Thu Sep 29, 2016 9:44 am

Re: exclude content of a txt/csv file when running script

Post by disguised sensei » Fri Sep 30, 2016 5:01 pm

cool job! Thank you for answering. I will try to find something.

update: no luck. Couldn't find any similar thread or script to my purpose. So if anyone would like to help, I'm all ears!
chivracq
Posts: 9179
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: exclude content of a txt/csv file when running script

Post by chivracq » Sat Oct 01, 2016 1:25 pm

disguised sensei wrote:cool job! Thank you for answering. I will try to find something.

update: no luck. Couldn't find any similar thread or script to my purpose. So if anyone would like to help, I'm all ears!
Beh no, your Case is unique, so you won't find an "exact" similar Thread, you need to understand the Principle and try stg by yourself, and sorry, but I won't be writing your whole Script for you, and I don't think anybody will either, so you need to go diving in the mud a bit.., beurk...! I'll help you further where you get stuck, even if pffff..., I don't do '.js', but if you go this way, I can still assist..., and there are a few other Advanced Users who only swear by '.js' and who'll be more than happy to follow up on this Thread, your Case is quite interesting...
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE').
- I don't even read the Qt if that (required) Info is not mentioned...!
- Script & URL help a lot for more "educated" Help...
disguised sensei
Posts: 32
Joined: Thu Sep 29, 2016 9:44 am

Re: exclude content of a txt/csv file when running script

Post by disguised sensei » Sat Oct 01, 2016 2:51 pm

here I tried something with iimGetExtract(). It looks more complicated but does nothing more than the code in my original post:

Code: Select all

var macro;
var ret;

macro ="CODE:";
macro +="SET !DATASOURCE URLs.txt"+"\n"
macro +="SET !ERRORIGNORE YES"+"\n";
macro +="TAG POS=1 TYPE=HTML ATTR=* EXTRACT=HTM"+"\n";
macro +="SET !DATASOURCE_LINE {{!LOOP}}"+"\n";

iimPlay(macro)

var text=iimGetExtract();

if(text.search("00016")!=-1)

{
ret = iimPlay("donothing");
}
else if (ret != -101) {

ret = iimPlay("openURL");

}
This here looks for the text.search and successfully finds it.

How do I change this so it searches through my text file instead of the text.search within the script?
disguised sensei
Posts: 32
Joined: Thu Sep 29, 2016 9:44 am

Re: exclude content of a txt/csv file when running script

Post by disguised sensei » Sat Oct 01, 2016 7:17 pm

whoever gets me done this script now gets 30 Euros via PayPal.
disguised sensei
Posts: 32
Joined: Thu Sep 29, 2016 9:44 am

Re: exclude content of a txt/csv file when running script

Post by disguised sensei » Sun Oct 02, 2016 5:36 am

anyone with an idea?
disguised sensei
Posts: 32
Joined: Thu Sep 29, 2016 9:44 am

Re: exclude content of a txt/csv file when running script

Post by disguised sensei » Sun Oct 02, 2016 7:01 am

this is an interesting place where you get to talk a lot with yourself ...
chivracq
Posts: 9179
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: exclude content of a txt/csv file when running script

Post by chivracq » Sun Oct 02, 2016 2:57 pm

disguised sensei wrote:whoever gets me done this script now gets 30 Euros via PayPal.
disguised sensei wrote:anyone with an idea?
disguised sensei wrote:this is an interesting place where you get to talk a lot with yourself ...
Ah-ah..., I wanted to start last night but I fell asleep, oops...!
Working on it...!

What I intend to do is have your Script first ask you (in a PROMPT) from which Row/Msg Nb on the Page you want to start with and the Macro will check the next 10 URL's and only open a New Tab (=> Max 10 Tabs, but you could easily extend it to 15 URL's/Tabs if you want) with that URL if it doesn't belong to the Exclude List.

For the PROMPT which might quickly become tedious to type each time 201, 211, 221, etc... for 2000 URL's, I could include a Default Value using the Clipboard or some mini-Temp File remembering the "msg01234" part of the last URL processed by the last time you ran the Macro.
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE').
- I don't even read the Qt if that (required) Info is not mentioned...!
- Script & URL help a lot for more "educated" Help...
Post Reply