Check if a CSS Selector exists on a webpage

Discussions and Tech Support related to website data extraction, screen scraping and data mining using iMacros.
Forum rules
iMacros EOL - Attention!

The renewal maintenance has officially ended for Progress iMacros effective November 20, 2023 and all versions of iMacros are now considered EOL (End-of-Life). The iMacros products will no longer be supported by Progress (aside from customer license issues), and these forums will also no longer be moderated from the Progress side.

Thank you again for your business and support.

Sincerely,
The Progress Team

Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information: CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
Post Reply
User avatar
WatermelonFish
Posts: 2
Joined: Fri Nov 12, 2021 7:53 pm

Check if a CSS Selector exists on a webpage

Post by WatermelonFish » Fri Nov 12, 2021 9:48 pm

Hi all,
Preface:
- I’m a beginner at coding, so your patience is appreciated :)
- Apologies if this isn’t the right forum group in which to post this question. I wasn’t quite sure where to put it.
- I’ve been looking for the answer to this question for several hours last night and this morning and couldn’t find it. If this type of question is already answered, I apologize for creating a redundant thread and would appreciate the link.
- I tried very hard to include all the information required, as mentioned in the “Please read this before reporting problems” thread. Please go easy on me if I neglected to include some important information. I’ll happily provide anything else that’s needed.

iMacros v10.1.0 for FF, 'Free', FF94 (v94.0.1 (64-bit)), Win10_ENG.

I want my script to be able to check to see if there’s a specific CSS Selector on a webpage. If it doesn’t exist, I want the script to continue. If it does exist, I want the script to extract the HREF/URL link from the selector, open up a new tab, go to the URL extracted from the selector, and then close the tab and return to the first tab.

The CSS selector in question is:

Code: Select all

body > div:nth-child(15) > a:nth-child(1)
I wrote this code out:

Code: Select all

SET !ERRORIGNORE YES
SET !TIMEOUT_STEP 1
TAG SELECTOR="body > div:nth-child(15) > a:nth-child(1)" EXTRACT=HREF
TAB OPEN
TAB T=2
URL GOTO={{!EXTRACT}}
WAIT SECONDS=1
TAB CLOSE
SET !EXTRACT NULL
TAB T=1
The problem with this code is that if the selector doesn’t exist, the script will open a new tab and then freeze (even though I have SET !ERRORIGNORE YES), presumably because it doesn’t know what to do with !EXTRACT, which is NULL. This is why I need the script to be able to check if the selector is on the page (unless there's another solution of which I'm not aware).

Additionally, I’m running the free version of iMacros, which only allows for three custom variables. I’m already using three variables to generate a random number earlier in the script:

Code: Select all

SET !VAR1 1
SET !VAR2 3
SET !VAR3 EVAL("var randomNumber=Math.floor(Math.random()*\"{{!VAR2}}\" +\"{{!VAR1}}\"); randomNumber;")
Is it possible to perform the check for the CSS selector without using any variables? If not, is it possible to generate a random number without the three variables so I can then use them for the check?

Thank you for reading and for your help. :D
Last edited by WatermelonFish on Sat Nov 13, 2021 7:08 pm, edited 1 time in total.
chivracq
Posts: 10301
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Check if a CSS Selector exists on a webpage

Post by chivracq » Sat Nov 13, 2021 11:55 am

WatermelonFish wrote:
Fri Nov 12, 2021 9:48 pm
Hi all,
Preface:
- I’m a beginner at coding, so your patience is appreciated :)
- Apologies if this isn’t the right forum group in which to post this question. I wasn’t quite sure where to put it.
- I’ve been looking for the answer to this question for several hours last night and this morning and couldn’t find it. If this type of question is already answered, I apologize for creating a redundant thread and would appreciate the link.
- I tried very hard to include all the information required, as mentioned in the “Please read this before reporting problems” thread. Please go easy on me if I neglected to include some important information. I’ll happily provide anything else that’s needed.

1. What version of iMacros are you using?
'VERSION BUILD=10101485 (FREE Version)

2. What operating system are you using? (please also specify language)
Windows 10 English

3. Which browser(s) are you using? (include version numbers)
Firefox 94.0.1 (64-bit)

4. Do the included demo macros work ok?
Yes

Questions 5-8 are irrelevant, I think

I want my script to be able to check to see if there’s a specific CSS Selector on a webpage. If it doesn’t exist, I want the script to continue. If it does exist, I want the script to extract the HREF/URL link from the selector, open up a new tab, go to the URL extracted from the selector, and then close the tab and return to the first tab.

The CSS selector in question is:

Code: Select all

body > div:nth-child(15) > a:nth-child(1)
I wrote this code out:

Code: Select all

SET !ERRORIGNORE YES
SET !TIMEOUT_STEP 1
TAG SELECTOR="body > div:nth-child(15) > a:nth-child(1)" EXTRACT=HREF
TAB OPEN
TAB T=2
URL GOTO={{!EXTRACT}}
WAIT SECONDS=1
TAB CLOSE
SET !EXTRACT NULL
TAB T=1
The problem with this code is that if the selector doesn’t exist, the script will open a new tab and then freeze (even though I have SET !ERRORIGNORE YES), presumably because it doesn’t know what to do with !EXTRACT, which is NULL. This is why I need the script to be able to check if the selector is on the page (unless there's another solution of which I'm not aware).

Additionally, I’m running the free version of iMacros, which only allows for three custom variables. I’m already using three variables to generate a random number earlier in the script:

Code: Select all

SET !VAR1 1
SET !VAR2 3
SET !VAR3 EVAL("var randomNumber=Math.floor(Math.random()*\"{{!VAR2}}\" +\"{{!VAR1}}\"); randomNumber;")
Is it possible to perform the check for the CSS selector without using any variables? If not, is it possible to generate a random number without the three variables so I can then use them for the check?

Thank you for reading and for your help. :D

Alright, mini-Compliment on the Good Quality of your OP... :D
(I'll only move your Thread to the 'Data Extraction' Sub-Forum where it "rather belongs" than the 'General' one, from the Functionality that you need/are using... (But the 'General' Sub-Forum is always "OK" if you are "unsure" where to open your Thread...))

"Simplified" Format you can also use to mention your FCI is: 8)

Code: Select all

iMacros v10.1.0 for FF, 'Free', FF94 (v94.0.1 (64-bit)), Win10_ENG.
>>>

Then alright, to answer your Scenario, hum..., yep indeed, I think the "URL GOTO=#EANF#" when the Link is not found, is "causing" the Pb...
The iMacros v10.x Versions for FF/CR only accept an URL in "http://" or "www", to "prevent" the Use of "Privileged" URL's, or the Script will simply hang... It's supposed to be "by Design", but I consider it a Bug..., or at least "a Bad Design", ah-ah...! :(

An "easy Workaround" would be, before going to the 'URL GOTO', to check if that Link exists, and using 'EVAL()', to compute the "real" URL, or to "transform" that "#EANF#" into an empty String or some "fake but real" URL, like "www.google.com" for example, like in:

Code: Select all

SET !ERRORIGNORE YES
SET !TIMEOUT_STEP 1

SET !EXTRACT NULL
TAG SELECTOR="body > div:nth-child(15) > a:nth-child(1)" EXTRACT=HREF
SET !VAR1 EVAL("var s='{{!EXTRACT}}'; var z; if(s!='#EANF#'){z=s;} else{z='';}; z;")

TAB OPEN
TAB T=2
'URL GOTO={{!EXTRACT}}
URL GOTO={{!VAR1}}
WAIT SECONDS=1
TAB CLOSE
TAB T=1
If "your Version" still has Pb with the empty String for the `URL GOTO`, then simply adapt the 'EVAL()' Statement for example to:

Code: Select all

SET !VAR1 EVAL("var s='{{!EXTRACT}}'; var z; if(s!='#EANF#'){z=s;} else{z='www.google.com';}; z;")
And if, when starting your Script, you always make sure you already have some 2nd Tab already open, then you don't need the `TAB OPEN` + `TAB CLOSE` Commands... And that "saves" 2 Lines, from the 50-Line Limitation for the 'Free' Version, ah-ah...! :P

>>>

And don't worry about the 3 Vars, it's a bit of a "fake" Limitation, easy to bypass, and only meant to nag Users...! :shock:

As you can see, I'm using '!VAR1' for the `EVAL()` Statement, to then reuse for the `URL GOTO` further up, but that's "fine", you only use '!VAR1' yourself to compute your Random Nb in '!VAR3', and you can reuse '!VAR1' after that, you don't need it anymore...

For such an "easy" Case, you could even hard-code the "1" + "3" Values in the `EVAL()` like:

Code: Select all

'SET !VAR3 EVAL("var randomNumber=Math.floor(Math.random()*\"{{!VAR2}}\" +\"{{!VAR1}}\"); randomNumber;")
SET !VAR3 EVAL("var randomNumber=Math.floor(Math.random()*3 + 1); randomNumber;")
Or you can also declare "all your Vars" in just 1 Var and use `split()` to access them. Here for example I declare 6 Vars...!: :twisted: :P

Code: Select all

SET !VAR1 _1_3_45_99_www.google.com_15_
SET !VAR3 EVAL("var v='{{!VAR1}}'; var x,y,z; x=v.split('_'); z=Math.floor(Math.random()*x[2] + x[1]); z;")
... And 'x[5]' would then be for "www.google.com"...! :idea:
(The Index for `split()` starts at "0", => hence the very first Underscore when declaring the List of Vars in '!VAR1' for the first Var (="1") to be 'x[1]'...)

But..., oh...!, hum...!, mini-Chance the "+ x[1]" in the `EVAL()` might be treated as a String because of the `split()`, and I'm not sure if the `Math.floor()` Method will "automatically" treat or convert it as/to a Number again, => always "safe" is to then "fake" multiply Numbers by 1:

Code: Select all

SET !VAR1 _1_3_45_99_www.google.com_15_
SET !VAR3 EVAL("var v='{{!VAR1}}'; var x,y,z; x=v.split('_'); z=Math.floor(Math.random()*x[2] + x[1]*1); z;")
Not needed for 'x[2]' because that Var is already used for a Multiplication...
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
User avatar
WatermelonFish
Posts: 2
Joined: Fri Nov 12, 2021 7:53 pm

Re: Check if a CSS Selector exists on a webpage

Post by WatermelonFish » Sat Nov 13, 2021 8:49 pm

It works beautifully! This is wonderful, thank you @chivracq :mrgreen:

For posterity, here's the working code (before beginning, have two tabs open and view the first tab):

Code: Select all

'VERSION BUILD=10101485
SET !ERRORIGNORE YES
SET !TIMEOUT_STEP 0

SET !VAR1 EVAL("var randomNumber=Math.floor(Math.random()*2 + 1); randomNumber;")
WAIT SECONDS={{!VAR1}}
SET !EXTRACT NULL
TAG ___Potential Button___ EXTRACT=HREF
SET !VAR2 EVAL("var s='{{!EXTRACT}}'; var z; if(s!='#EANF#'){z=s;} else{z='www.google.com';}; z;")
TAB T=2
URL GOTO={{!VAR2}}
WAIT SECONDS={{!VAR1}}
TAB T=1
TAG ___Button 1___
WAIT SECONDS={{!VAR1}}
TAG ___Button 2___
Thanks to your tips, I have shortened the code such that I can have four repetitions per macro cycle.

Code: Select all

SET !ERRORIGNORE YES
SET !TIMEOUT_STEP 0
SET !VAR1 EVAL("var randomNumber=Math.floor(Math.random()*2 + 1); randomNumber;")
WAIT SECONDS={{!VAR1}}
SET !EXTRACT NULL
TAG ___Potential Button___ EXTRACT=HREF
SET !VAR2 EVAL("var s='{{!EXTRACT}}'; var z; if(s!='#EANF#'){z=s;} else{z='www.google.com';}; z;")
TAB T=2
URL GOTO={{!VAR2}}
WAIT SECONDS={{!VAR1}}
TAB T=1
TAG ___Button 1___
WAIT SECONDS={{!VAR1}}
TAG ___Button 2___
SET !VAR1 EVAL("var randomNumber=Math.floor(Math.random()*2 + 1); randomNumber;")
WAIT SECONDS={{!VAR1}}
SET !EXTRACT NULL
TAG ___Potential Button___ EXTRACT=HREF
SET !VAR2 EVAL("var s='{{!EXTRACT}}'; var z; if(s!='#EANF#'){z=s;} else{z='www.google.com';}; z;")
TAB T=2
URL GOTO={{!VAR2}}
WAIT SECONDS={{!VAR1}}
TAB T=1
TAG ___Button 1___
WAIT SECONDS={{!VAR1}}
TAG ___Button 2___
SET !VAR1 EVAL("var randomNumber=Math.floor(Math.random()*2 + 1); randomNumber;")
WAIT SECONDS={{!VAR1}}
SET !EXTRACT NULL
TAG ___Potential Button___ EXTRACT=HREF
SET !VAR2 EVAL("var s='{{!EXTRACT}}'; var z; if(s!='#EANF#'){z=s;} else{z='www.google.com';}; z;")
TAB T=2
URL GOTO={{!VAR2}}
WAIT SECONDS={{!VAR1}}
TAB T=1
TAG ___Button 1___
WAIT SECONDS={{!VAR1}}
TAG ___Button 2___
SET !VAR1 EVAL("var randomNumber=Math.floor(Math.random()*2 + 1); randomNumber;")
WAIT SECONDS={{!VAR1}}
SET !EXTRACT NULL
TAG ___Potential Button___ EXTRACT=HREF
SET !VAR2 EVAL("var s='{{!EXTRACT}}'; var z; if(s!='#EANF#'){z=s;} else{z='www.google.com';}; z;")
TAB T=2
URL GOTO={{!VAR2}}
WAIT SECONDS={{!VAR1}}
TAB T=1
TAG ___Button 1___
WAIT SECONDS={{!VAR1}}
TAG ___Button 2___
Excellent :lol:
chivracq
Posts: 10301
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Check if a CSS Selector exists on a webpage

Post by chivracq » Sat Nov 13, 2021 10:26 pm

WatermelonFish wrote:
Sat Nov 13, 2021 8:49 pm
It works beautifully! This is wonderful, thank you @chivracq :mrgreen:

For posterity, here's the working code (before beginning, have two tabs open and view the first tab):

Code: Select all

'VERSION BUILD=10101485
SET !ERRORIGNORE YES
SET !TIMEOUT_STEP 0

SET !VAR1 EVAL("var randomNumber=Math.floor(Math.random()*2 + 1); randomNumber;")
WAIT SECONDS={{!VAR1}}
SET !EXTRACT NULL
TAG ___Potential Button___ EXTRACT=HREF
SET !VAR2 EVAL("var s='{{!EXTRACT}}'; var z; if(s!='#EANF#'){z=s;} else{z='www.google.com';}; z;")
TAB T=2
URL GOTO={{!VAR2}}
WAIT SECONDS={{!VAR1}}
TAB T=1
TAG ___Button 1___
WAIT SECONDS={{!VAR1}}
TAG ___Button 2___
Thanks to your tips, I have shortened the code such that I can have four repetitions per macro cycle.

Code: Select all

SET !ERRORIGNORE YES
SET !TIMEOUT_STEP 0
SET !VAR1 EVAL("var randomNumber=Math.floor(Math.random()*2 + 1); randomNumber;")
WAIT SECONDS={{!VAR1}}
SET !EXTRACT NULL
TAG ___Potential Button___ EXTRACT=HREF
SET !VAR2 EVAL("var s='{{!EXTRACT}}'; var z; if(s!='#EANF#'){z=s;} else{z='www.google.com';}; z;")
TAB T=2
URL GOTO={{!VAR2}}
WAIT SECONDS={{!VAR1}}
TAB T=1
TAG ___Button 1___
WAIT SECONDS={{!VAR1}}
TAG ___Button 2___
SET !VAR1 EVAL("var randomNumber=Math.floor(Math.random()*2 + 1); randomNumber;")
WAIT SECONDS={{!VAR1}}
SET !EXTRACT NULL
TAG ___Potential Button___ EXTRACT=HREF
SET !VAR2 EVAL("var s='{{!EXTRACT}}'; var z; if(s!='#EANF#'){z=s;} else{z='www.google.com';}; z;")
TAB T=2
URL GOTO={{!VAR2}}
WAIT SECONDS={{!VAR1}}
TAB T=1
TAG ___Button 1___
WAIT SECONDS={{!VAR1}}
TAG ___Button 2___
SET !VAR1 EVAL("var randomNumber=Math.floor(Math.random()*2 + 1); randomNumber;")
WAIT SECONDS={{!VAR1}}
SET !EXTRACT NULL
TAG ___Potential Button___ EXTRACT=HREF
SET !VAR2 EVAL("var s='{{!EXTRACT}}'; var z; if(s!='#EANF#'){z=s;} else{z='www.google.com';}; z;")
TAB T=2
URL GOTO={{!VAR2}}
WAIT SECONDS={{!VAR1}}
TAB T=1
TAG ___Button 1___
WAIT SECONDS={{!VAR1}}
TAG ___Button 2___
SET !VAR1 EVAL("var randomNumber=Math.floor(Math.random()*2 + 1); randomNumber;")
WAIT SECONDS={{!VAR1}}
SET !EXTRACT NULL
TAG ___Potential Button___ EXTRACT=HREF
SET !VAR2 EVAL("var s='{{!EXTRACT}}'; var z; if(s!='#EANF#'){z=s;} else{z='www.google.com';}; z;")
TAB T=2
URL GOTO={{!VAR2}}
WAIT SECONDS={{!VAR1}}
TAB T=1
TAG ___Button 1___
WAIT SECONDS={{!VAR1}}
TAG ___Button 2___
Excellent :lol:

Alright...!, Thanks for sharing your Final Script applying my "Tips", and glad I could help..., ... which is always a Pleasure with Users putting some Quality in their Thread(s) like you did...! :D

>>>

Hum, and I would have a few more "Tips", ah-ah...! :shock: :twisted:

If you are "desperate", you could "squeeze" 5 Blocks in the Script per Run, ah-ah...! :P

- Each Block contains currently 12 Lines, but it starts and finishes with a 'WAIT' Statement. Starting a Script (or a Block if repeated in a Script) has usually no Use, and in your Case, last 'WAIT' from Block_1 + first 'WAIT' from Block_2 gives 'WAIT' + WAIT' = the Script doesn't do anything but "waiting", => 1 'WAIT' per Block can be removed... :idea:
=> 11 Lines per Block...! 8)

- Maybe you don't need for each Block to re-compute the Random Value for all 'WAIT' Statements and can keep using the same Value for a whole Run...
=> -4 Lines.
Or maybe recompute only once that Random Value in the middle of the Script for Block_3 or Block_4... :idea:
=> Still saving -3 Lines.

- The very first "SET !EXTRACT NULL" (in Block_1) is not needed, the '!EXTRACT' Var is already empty at that moment...
=> -1 Line.

- If "SET !ERRORIGNORE YES" is "meant" for the "Potential Button" that might not be present, the `EXTRACT` Command on that Button/Link makes sure that the Script will never abort on that Line...
=> -1 Line.

- And hum, each Block contains "a lot" of 'WAIT' Statements, nearly between every single `TAG` or `URL GOTO` Command, maybe one "SET !PLAYBACKDELAY {{!VAR1}}" Statement per Block, or for the whole Script, or maybe x2 in the Script would achieve more or less the same Result like all the `WAIT` Statements... (And you can also use Decimal Values for `!PLAYBACKDELAY`...) :idea:
=> -9 / -8 / -7 Lines...
=> About 9 Lines per Block, and you now can squeeze 5 Blocks in the Script, ah-ah...! 8) :P

+ All "SET !EXTRACT NULL" could also be removed, but the `EVAL()` Statements for Blocks_[2-5] would need to be adapted to always take only the last 'EXTRACT' into account...
=> -4 Lines.

>

One more Tip, but that would add +1 Line per Block, is that I don't find the 'BODY>DIV[15]' very "reliable" for your 'SELECTOR', this will easily get broken on most Sites, as soon as any "Content" changes on the Page, 'DIV[15]' will quickly become 'DIV[14]' or 'DIV[16]', ... more reliable would be to try to use the 'SELECTOR' Mode with ID's, if possible, => to "shorten" the CSS Path..., or even more reliable, => to use 'Relative Positioning'... :idea:
(But 'R-POS' requires 2 Lines, 1 for the 'Anchor', and 1 for the "real" `TAG`...)
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
chivracq
Posts: 10301
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Check if a CSS Selector exists on a webpage

Post by chivracq » Sun Nov 14, 2021 12:56 am

For info, I moved the Thread to the 'Data Extraction' Sub-Forum, like i had mentioned I would do, in my first Reply in this thread... :|
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
Post Reply