Extract XPATH with Personal version

For trouble installing the software or activating your license, please open a support case. Freeware and Personal Edition users can also post questions here.
Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information:CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
Post Reply
harpyseal
Posts: 7
Joined: Tue Sep 01, 2015 12:41 am

Extract XPATH with Personal version

Post by harpyseal » Fri Jul 19, 2019 6:19 pm

I have a simple macro which extracts text using XPATH.
This works fine on my Professional set up, but my client downloaded the 30-day Personal version which, according to the Features Overview, has full web-extraction capabilities but it doesn't work.

iMacros support tells me if I BUY the Personal version it will work. This seems fishy to me. How do you have a 30-day trial with missing features?
And, how do you say the Personal version has full web-extraction capabilities if it's not going to allow extracting text from XPATH, and you need to buy the Professional for this one thing?
chivracq
Posts: 8424
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Extract XPATH with Personal version

Post by chivracq » Sat Jul 20, 2019 6:22 pm

harpyseal wrote:
Fri Jul 19, 2019 6:19 pm
I have a simple macro which extracts text using XPATH.
This works fine on my Professional set up, but my client downloaded the 30-day Personal version which, according to the Features Overview, has full web-extraction capabilities but it doesn't work.

iMacros support tells me if I BUY the Personal version it will work. This seems fishy to me. How do you have a 30-day trial with missing features?
And, how do you say the Personal version has full web-extraction capabilities if it's not going to allow extracting text from XPATH, and you need to buy the Professional for this one thing?

Hum, it's not very "clear" about which 'PE' Version and which Browser(s) you are talking...!? :?

Only iMB has a 30-day Trial Version, there is no Trial Version for the 3 Add-ons for IE/FF/CR, only the 'Free' + 'PE' Versions, and yep, the 'Free' Versions are "limited", but there is no Limitation related to the 'TAG XPATH' Mode as far as I know...

You are not talking about the 'TAG SELECTOR' Mode by any chance...? As this Mode had been introduced in v10.0.x for CR, and I guess it has since also been implemented in iMB v12.5, but I'm not completely sure if it has also been implemented in v10.0.2 for FF.

No Script posted, so it's a bit difficult to know exactly what you are talking about..., and your "but it doesn't work" is a bit vague also... :?
But the 'TAG XPATH' + 'EXTRACT' Mechanism should work in absolutely all Versions, afaik...
And I don't know of any Bug either about it, in any Version(s) (currently in use)...

Only the 'SAVEAS TYPE=EXTRACT' Functionality is indeed limited (= not included) in the 'Free' Versions for FF + CR. (v12.5 'Free' for IE not impacted.) You then need the 'FIO' Module for Local File Access, which is indeed only included in the 'PE' Version for those 2 Browsers. And there is also no 'FIO' Module for MacOS nor Linux (yet...), only Win32/64 as OS is supported.

>>>

You didn't follow up anymore btw on your previous Thread to finish it a bit "correctly" and to mention if (and how?) you managed to find a Solution...? :cry:
Hum, and same Remark also about your very first Thread where you never followed up at all... :shock:
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all 3 Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
Tom, Tech Support
Posts: 3491
Joined: Mon May 31, 2010 4:59 pm

Re: Extract XPATH with Personal version

Post by Tom, Tech Support » Mon Jul 22, 2019 8:49 am

harpyseal wrote:
Fri Jul 19, 2019 6:19 pm
I have a simple macro which extracts text using XPATH.
This works fine on my Professional set up, but my client downloaded the 30-day Personal version which, according to the Features Overview, has full web-extraction capabilities but it doesn't work.

iMacros support tells me if I BUY the Personal version it will work. This seems fishy to me. How do you have a 30-day trial with missing features?
And, how do you say the Personal version has full web-extraction capabilities if it's not going to allow extracting text from XPATH, and you need to buy the Professional for this one thing?
In what browser specifically and with which version of iMacros is your client testing? Have they tried the demo FillForm-XPath macro that installs with all versions of iMacros? Any one of the TAG commands in that macro can be modified easily to perform an extraction by replacing the CONTENT parameter with e.g. EXTRACT=HTM and then running the macro to see if the extraction is successful.

A little bit of troubleshooting before posting and providing specific information when you do make a post goes a long way toward getting meaningful help faster!
Regards,

Tom, iMacros Support
harpyseal
Posts: 7
Joined: Tue Sep 01, 2015 12:41 am

Re: Extract XPATH with Personal version

Post by harpyseal » Tue Jul 23, 2019 6:29 pm

Oh! Sorry! I don't seem to be getting replies to my posts! Thank you so much for your info and helping me out!

Here's the macro
VERSION BUILD=12.5.503.8802
TAB T=1
TAB CLOSEALLOTHERS
SET !DATASOURCE "address.csv"
SET !LOOP 1
SET !DATASOURCE_LINE {{!LOOP}}
SET !VAR1 {{!COL2}}

URL GOTO = {{!COL1}}
SET !TIMEOUT_STEP 7
SET !EXTRACT_TEST_POPUP NO
TAG XPATH="//*[@id='pane']/div/div[1]/div/div/div[3]/div[1]/div[2]/div/div[1]/span[2]/ul/li/span[2]/span[1]/button" EXTRACT=TXT
TAG XPATH="//*[@id='pane']/div/div[1]/div/div/div[3]/div[1]/div[2]/div/div[2]/span[1]/span[1]/button" EXTRACT=TXT
ADD !EXTRACT {{!VAR1}}
SAVEAS TYPE=EXTRACT FOLDER=* FILE="saveas.csv"
WAIT SECONDS=1

The URLs are just concatenations of company name and addresses, searching in google maps, like:
"http://local.google.com/maps?q=TOTAL FITNESS LLC, 71717 29 PALMS HWY,TWENTYNINE PALMS,CA,92277,"

When I run this on the Personal version the macro hangs when it gets to the TAG XPATH line. If I remove the two TAG XPATH lines and just extract VAR1 then ti works.
When I run this on the Professional version it works fine.

I'm not trying to use the free browser add-ons. I'd like to get my client to purchase the Personal version because he's going to balk at the price of the Professional.

This is the set up info

Microsoft Windows Server 2016 Standard Evaluation
Microsoft Windows NT 6.3.9600.0
64-bit Operating System
Installed UI Culture: English (United States)
CLR version 4.0.30319.42000
Internet Explorer version 11.2999.14393.0
Enhanced Protected Mode = False, 64-bit processes for EPM = False
BrowserEmulation Mode = 7
iMacros Browser (x64) version 12.5.2018.1105
Licensed Product: Trial Version
License Type: 30-day trial (24 days left)
------------------------------------------------------------------------------------------
Google Chrome version 75.0.3770.142
------------------------------------------------------------------------------------------
iMacros File Access is not installed
------------------------------------------------------------------------------------------
Auxiliary Libraries:
iMacros Direct Screen [iimds.dll] version 12.0.0.19
iMacros Fx&Cr Image Recognition (x86) [iimIRm.dll] version 12.0.503.8173
iMacros Image Recognition (x86) [ImageRecognitionLibrary.dll] version 12.0.503.8173
iMacros Dialog Manager (x86) [imsys.dll] version 12.5.503.8802
iMacros Fx&Cr Image Recognition (x86) [iimIRm.dll] version 12.0.503.8173
iMacros Direct Screen [iimds.dll] version 12.0.0.19
iMacros Fx&Cr Image Recognition (x64) [iimIRm.dll] version 12.0.503.8173
iMacros Image Recognition (x64) [ImageRecognitionLibrary.dll] version 12.0.503.8173
iMacros Dialog Manager (x86) [imsys.dll] version 12.5.2018.1105
iMacros Fx&Cr Image Recognition (x64) [iimIRm.dll] version 12.0.503.8173
------------------------------------------------------------------------------------------
Scripting Interface Libraries:
iMacros Scripting Connector (x86) [iimConnector.dll] version 12.0.5.163
iMacros Scripting Connector (x64) [iimConnector64.dll] version 12.0.5.163
------------------------------------------------------------------------------------------
User Settings:
FolderLogs = C:\Users\angie\Documents\iMacros\Logs
FolderDownloads = C:\Users\angie\Documents\iMacros\Downloads
NewWindowBehavior = Tabs
FolderDataSources = C:\Users\angie\Documents\iMacros\DataSources
FolderMacros = C:\Users\angie\Documents\iMacros\Macros
InterfaceType =
InstalledSamples = True
chivracq
Posts: 8424
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Extract XPATH with Personal version

Post by chivracq » Thu Jul 25, 2019 6:50 am

harpyseal wrote:
Tue Jul 23, 2019 6:29 pm
Oh! Sorry! I don't seem to be getting replies to my posts! Thank you so much for your info and helping me out!

Here's the macro

Code: Select all

VERSION BUILD=12.5.503.8802
TAB T=1
TAB CLOSEALLOTHERS
SET !DATASOURCE "address.csv"
SET !LOOP 1
SET !DATASOURCE_LINE {{!LOOP}}
SET !VAR1 {{!COL2}}

URL GOTO = {{!COL1}}
SET !TIMEOUT_STEP 7
SET !EXTRACT_TEST_POPUP NO
TAG XPATH="//*[@id='pane']/div/div[1]/div/div/div[3]/div[1]/div[2]/div/div[1]/span[2]/ul/li/span[2]/span[1]/button" EXTRACT=TXT
TAG XPATH="//*[@id='pane']/div/div[1]/div/div/div[3]/div[1]/div[2]/div/div[2]/span[1]/span[1]/button" EXTRACT=TXT
ADD !EXTRACT {{!VAR1}} 
SAVEAS TYPE=EXTRACT FOLDER=* FILE="saveas.csv"
WAIT SECONDS=1
The URLs are just concatenations of company name and addresses, searching in google maps, like:

Code: Select all

"http://local.google.com/maps?q=TOTAL FITNESS LLC, 71717 29 PALMS HWY,TWENTYNINE PALMS,CA,92277,"
When I run this on the Personal version the macro hangs when it gets to the TAG XPATH line. If I remove the two TAG XPATH lines and just extract VAR1 then ti works.
When I run this on the Professional version it works fine.

I'm not trying to use the free browser add-ons. I'd like to get my client to purchase the Personal version because he's going to balk at the price of the Professional.

This is the set up info

Code: Select all

Microsoft Windows Server 2016 Standard Evaluation
Microsoft Windows NT 6.3.9600.0
64-bit Operating System
Installed UI Culture: English (United States)
CLR version 4.0.30319.42000
Internet Explorer version 11.2999.14393.0
Enhanced Protected Mode = False, 64-bit processes for EPM = False
BrowserEmulation Mode = 7
iMacros Browser (x64) version 12.5.2018.1105
Licensed Product: Trial Version
License Type: 30-day trial (24 days left)
------------------------------------------------------------------------------------------
Google Chrome version 75.0.3770.142
------------------------------------------------------------------------------------------
iMacros File Access is not installed
------------------------------------------------------------------------------------------
Auxiliary Libraries:
iMacros Direct Screen [iimds.dll] version 12.0.0.19
iMacros Fx&Cr Image Recognition (x86) [iimIRm.dll] version 12.0.503.8173
iMacros Image Recognition (x86) [ImageRecognitionLibrary.dll] version 12.0.503.8173
iMacros Dialog Manager (x86) [imsys.dll] version 12.5.503.8802
iMacros Fx&Cr Image Recognition (x86) [iimIRm.dll] version 12.0.503.8173
iMacros Direct Screen [iimds.dll] version 12.0.0.19
iMacros Fx&Cr Image Recognition (x64) [iimIRm.dll] version 12.0.503.8173
iMacros Image Recognition (x64) [ImageRecognitionLibrary.dll] version 12.0.503.8173
iMacros Dialog Manager (x86) [imsys.dll] version 12.5.2018.1105
iMacros Fx&Cr Image Recognition (x64) [iimIRm.dll] version 12.0.503.8173
------------------------------------------------------------------------------------------
Scripting Interface Libraries:
iMacros Scripting Connector (x86) [iimConnector.dll] version 12.0.5.163
iMacros Scripting Connector (x64) [iimConnector64.dll] version 12.0.5.163
------------------------------------------------------------------------------------------
User Settings:
FolderLogs = C:\Users\angie\Documents\iMacros\Logs
FolderDownloads = C:\Users\angie\Documents\iMacros\Downloads
NewWindowBehavior = Tabs
FolderDataSources = C:\Users\angie\Documents\iMacros\DataSources
FolderMacros = C:\Users\angie\Documents\iMacros\Macros
InterfaceType = 
InstalledSamples = True
My 2cts again, I had "expected" TechSup to react/reply... :o

Well, still unclear to me in which 'PE' Version you do your Testing, @OP, I see some CR75 mentioned in your "Setup Info", but "iMacros File Access is not installed" also, meaning '{{!COLn}}' + 'SAVEAS TYPE EXTRACT' won't work, so I don't understand your "and just extract VAR1 then ti works."... :?

But OK, I had a look at your Script "yesterday" (first time btw I see an URL with Spaces, and that works, hum...!, OK...), but hum, my first Reaction is that your both 'TAG XPATH' Statements are very "hazardous", and way too long to be "reliable"... :shock: They won't work anymore upon the slightest Change of Content/Layout/Functionality from the Site... Oh...!, and the Site is Google, ah-ah...!, meant as a Joke maybe...!?, every week they change "stg" to all their Sites... :P

OK, maybe you want to go on some "Crusade" with TechSup that 'XPATH' maybe doesn't work in the still undetermined 'PE' Version/Environment you are doing your Testing in, hum OK, could be..., (and I won't interfere in that Case...), but..., pfff, I would have some much easier and much more straightforward Solution for you using the "standard" 'TAG POS' Mode, if you are interested... :idea:

Your first Statement anyway could only trigger some "ambiguous XPATH" RuntimeError for me (tested on iMacros for FF v8.8.2 + PM v26.3.3 + Win10_x64), I guess you were trying to extract the "66" from that Page...?
+ => "Gym" from the 2nd 'EXTRACT'...?
(No Comments in your Script, so it's not "clear" what exactly you want to extract... :? )
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all 3 Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
Tom, Tech Support
Posts: 3491
Joined: Mon May 31, 2010 4:59 pm

Re: Extract XPATH with Personal version

Post by Tom, Tech Support » Thu Jul 25, 2019 3:12 pm

Tech Support has been handling this case. The issue is that the first TAG XPATH command freezes when playing the macro in IE (it does not matter which license is used, it is a browser-specific issue). The macro runs without issue in the iMacros browser, and also runs in Chrome and Firefox although the XPATH needs to be updated to extract the correct element in those browsers.

Tech support is also looking into changing the macro to use TAG SELECTOR instead of XPATH as a workaround for running it in IE.
Regards,

Tom, iMacros Support
harpyseal
Posts: 7
Joined: Tue Sep 01, 2015 12:41 am

Re: Extract XPATH with Personal version

Post by harpyseal » Thu Jul 25, 2019 5:48 pm

Thank you chivracq! I guess TechSup is also chiming in now so I hope we get this resolved. Thank you Tom for answering, am super happy to hear from you.

- By "Personal" I mean the $99 edition on the Features Comparison Chart on iMacros website. The page is divided into Free Browser Add-on, Personal Edition, Professional Edition, Enterprise. etc.
- I also was stumped by the fact "iMacros File Access is not installed" but when I run this macro (with above settings) using the iMacros browser, it works.
What I was trying to say (and wasn't clear - sorry) is that this macro fails using IE (since the Personal version doesn't have the iMacros browser) but running the demo files "FillForm_XPath" and "Extract Table" on IE both work fine.
After my macro failed (it just hangs on that first TAG XPATH line) I commented out the two XPATH lines and asked it to return just the VAR1 values to a csv, (The value for VAR1 comes from the second column of my datasource) and that worked, so I suspect the problem is with extracting from XPATH to csv file using IE.

- I understand issue with my XPATH being "hazardous" and "way too long" :D but that's the only way I knew how to do it, if can please help me with your idea of using the "Standard' TAG POS Mode I would be grateful. Or TAG SELECTOR as Tom suggests.
Yes, I'm trying to extract the "66" (number of reviews) and "Gym" (type of company) from the google search. Learning how to write the script correctly would be helpful!

- I cannot test using Chrome without a paid license. Is it really true that this will work if I get my client to buy a license? I can try to convince him..

Thanks everyone so much.

Angie
Tom, Tech Support
Posts: 3491
Joined: Mon May 31, 2010 4:59 pm

Re: Extract XPATH with Personal version

Post by Tom, Tech Support » Thu Jul 25, 2019 5:52 pm

iMacros for IE does not handle XPATH as well as the other versions. However, the XPATH can be easily converted to SELECTOR commands, and these work without any problem in IE:

Code: Select all

TAG SELECTOR="#pane>DIV>DIV>DIV>DIV>DIV:nth-of-type(3)>DIV>DIV:nth-of-type(2)>DIV>DIV>SPAN:nth-of-type(2)>UL>LI>SPAN:nth-of-type(2)>SPAN>BUTTON" EXTRACT=TXT
TAG SELECTOR="#pane>DIV>DIV>DIV>DIV>DIV:nth-of-type(3)>DIV>DIV:nth-of-type(2)>DIV>DIV:nth-of-type(2)>SPAN>SPAN>BUTTON" EXTRACT=TXT
Regards,

Tom, iMacros Support
chivracq
Posts: 8424
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Extract XPATH with Personal version

Post by chivracq » Thu Jul 25, 2019 7:35 pm

Yep OK, so @OP, you were running your Test in IE, which indeed explains why the '{{!COLn}}' were working, as Local File Access is still supported in v12.5 for IE 'Free'. (Only v10.0.x for FF/CR need the FIO-Module which is only included in the 'PE' Versions for both Browsers...)

Hum, "interesting" to see that different Browsers handle 'XPATH' differently apparently, or at least for such long 'XPATH''s...
=> Even if @Tom's Solution using the 'TAG SELECTOR' Mode will work, I still find it as "hazardous" as the ones using 'XPATH' because of the Length of the 'Locator', and those Statements will probably easily get broken each time some "Functionality" changes on the Page...

From a 2nd Look at the Page, here is another much simpler Script, and more reliable I would think:

Code: Select all

VERSION BUILD=8820413 RECORDER=FX
SET !EXTRACT_TEST_POPUP NO
TAB T=1
'URL GOTO=https://www.google.com/maps/place/Total+Fitness/@34.1342529,-116.0978325,17z/data=!4m5!3m4!1s0x80dac6d056278879:0xc0bb40a84fc0b73d!8m2!3d34.1342529!4d-116.0978325

SET !EXTRACT NULL
TAG POS=1 TYPE=SPAN ATTR=CLASS:"section-star-display"&&TXT:* EXTRACT=TXT
SET Rating {{!EXTRACT}}

SET !EXTRACT NULL
TAG POS=1 TYPE=BUTTON ATTR=TXT:*review* EXTRACT=TXT
SET Nb_Reviews1 {{!EXTRACT}}

SET !EXTRACT NULL
TAG POS=1 TYPE=BUTTON ATTR=CLASS:"widget-pane-link"&&TXT:* EXTRACT=TXT
SET Nb_Reviews2 {{!EXTRACT}}

SET !EXTRACT NULL
TAG POS=5 TYPE=BUTTON ATTR=CLASS:"widget-pane-link"&&TXT:* EXTRACT=TXT
SET Category {{!EXTRACT}}

'PROMPT _{{!EXTRACT}}_
PROMPT Rating:<SP>_{{Rating}}_<BR>Nb_Reviews1:<SP>_{{Nb_Reviews1}}_<BR>Nb_Reviews2:<SP>_{{Nb_Reviews2}}_<BR>Category:<SP>_{{Category}}_
... which will display the following in the 'PROMPT':
Rating: _4.3_
Nb_Reviews1: _68 reviews_
Nb_Reviews2: _(68)_
Category: _Gym_
(Tested in iMacros for FF v8.8.2, PM v26.3.3 (=FF47), Win10_x64.)

Oh...!, but hum, @OP, if you want to test in any of the 3 'Free' Versions (IE/FF/CR), you'll need to replace/rename all the Vars I used with '!VAR[1-3]' as User-defined Vars are not supported in the 'Free' Versions..., which would give stg like:

Code: Select all

VERSION BUILD=8820413 RECORDER=FX
SET !EXTRACT_TEST_POPUP NO
TAB T=1
'URL GOTO=https://www.google.com/maps/place/Total+Fitness/@34.1342529,-116.0978325,17z/data=!4m5!3m4!1s0x80dac6d056278879:0xc0bb40a84fc0b73d!8m2!3d34.1342529!4d-116.0978325

SET !EXTRACT NULL
TAG POS=1 TYPE=SPAN ATTR=CLASS:"section-star-display"&&TXT:* EXTRACT=TXT
'SET Rating {{!EXTRACT}}
SET !VAR1 {{!EXTRACT}}
PROMPT Bonus...!<BR>Rating:<SP>_{{!VAR1}}_

SET !EXTRACT NULL
TAG POS=1 TYPE=BUTTON ATTR=TXT:*review* EXTRACT=TXT
'SET Nb_Reviews1 {{!EXTRACT}}
SET !VAR1 {{!EXTRACT}}

SET !EXTRACT NULL
TAG POS=1 TYPE=BUTTON ATTR=CLASS:"widget-pane-link"&&TXT:* EXTRACT=TXT
'SET Nb_Reviews2 {{!EXTRACT}}
SET !VAR2 {{!EXTRACT}}

SET !EXTRACT NULL
TAG POS=5 TYPE=BUTTON ATTR=CLASS:"widget-pane-link"&&TXT:* EXTRACT=TXT
'SET Category {{!EXTRACT}}
SET !VAR3 {{!EXTRACT}}

'PROMPT _{{!EXTRACT}}_
'PROMPT Rating:<SP>_{{Rating}}_<BR>Nb_Reviews1:<SP>_{{Nb_Reviews1}}_<BR>Nb_Reviews2:<SP>_{{Nb_Reviews2}}_<BR>Category:<SP>_{{Category}}_
PROMPT Nb_Reviews1:<SP>_{{!VAR1}}_<BR>Nb_Reviews2:<SP>_{{!VAR2}}_<BR>Category:<SP>_{{!VAR3}}_
Yep, this works also, you get the Data displayed in 2 'PROMPT''s, with the 'Rating' as "extra"..., and 2 Solutions for the 'Nb of Reviews'..., you can choose which one you prefer... 8)
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all 3 Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
harpyseal
Posts: 7
Joined: Tue Sep 01, 2015 12:41 am

Re: Extract XPATH with Personal version

Post by harpyseal » Thu Jul 25, 2019 10:45 pm

YAY. All of these solutions work perfectly in IE, and I love how neat and compact and "unhazardous" the extract line is.
Thank you both so much!
I'm going to go ahead and recommend my client purchase the Personal version and he'll be good to go. :P
chivracq
Posts: 8424
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Extract XPATH with Personal version

Post by chivracq » Fri Jul 26, 2019 4:21 am

harpyseal wrote:
Thu Jul 25, 2019 10:45 pm
YAY. All of these solutions work perfectly in IE, and I love how neat and compact and "unhazardous" the extract line is.
Thank you both so much!
I'm going to go ahead and recommend my client purchase the Personal version and he'll be good to go. :P
:D
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all 3 Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
Post Reply