Using EVENT to extract

Discussions and Tech Support related to website data extraction, screen scraping and data mining using iMacros.
Forum rules
iMacros EOL - Attention!

The renewal maintenance has officially ended for Progress iMacros effective November 20, 2023 and all versions of iMacros are now considered EOL (End-of-Life). The iMacros products will no longer be supported by Progress (aside from customer license issues), and these forums will also no longer be moderated from the Progress side.

Thank you again for your business and support.

Sincerely,
The Progress Team

Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information: CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
Post Reply
zirjeo
Posts: 82
Joined: Fri Oct 28, 2016 1:49 am

Using EVENT to extract

Post by zirjeo » Mon Sep 10, 2018 10:54 pm

VERSION BUILD=11.5.499.3066 IE 11, Windows 10

When using..

Code: Select all

EVENT TYPE=CLICK SELECTOR="#TaxHistory_Table>TBODY>TR>TD:nth-of-type(3)" BUTTON=0
TAG POS=R1 TYPE=TD ATTR=TXT:* EXTRACT=TXT
Is there any way to use EVENT for the 2nd line instead of TAG? Using TAG it's very slow to extract the text. Or maybe there's another faster way?

Thanks
chivracq
Posts: 10301
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Using EVENT to extract

Post by chivracq » Mon Sep 10, 2018 11:19 pm

zirjeo wrote:

Code: Select all

VERSION BUILD=11.5.499.3066  IE 11, Windows 10
When using..

Code: Select all

EVENT TYPE=CLICK SELECTOR="#TaxHistory_Table>TBODY>TR>TD:nth-of-type(3)" BUTTON=0
TAG POS=R1 TYPE=TD ATTR=TXT:* EXTRACT=TXT
Is there any way to use EVENT for the 2nd line instead of TAG? Using TAG it's very slow to extract the text. Or maybe there's another faster way?

Thanks
Hum..., are you sure your 'POS=R1' is working on an Anchor as 'EVENT'...!? I think I had already tried "once" and it was not working... :?
You don't post any URL, so I would have to redo my whole Testing...

But if "that" works, there is no reason why "it" would be slow for the 'EXTRACT', the 'TAG' Mode is actually much quicker than the 'EVENT' Mode...
... If at least you use the "correct" Settings/Values for all '!TIMEOUT_xxx' Settings (especially for '#EANF#'), can't check that if you don't post your whole Script... :roll:

And, yep there is a "way" to extract "Content" using the 'EVENT' Mode but it's a bit cumbersome, but I think it wouldn't work on IE as it requires the 'MODIFIERS' Parameter which I think is not supported on IE...

>>>

And if you posted your URL and Full Reqs, I would probably find you a "better" and more efficient/reliable Solution...

Strikes me that for maybe 80% of the Threads/Qt's on the Forum, you Guys already have some "Idea" on how to implement it (but usually get stuck at some unneeded intermediary Step), "Idea" which in most of the "Cases" turns out to be pretty cumbersome as you don't "focus" on your "final Goal" and in those 80% Cases there will be a much simpler and directer Solution..., while on the contrary when indeed some "cumbersome" and "Creative" Thinking is needed to achieve that "final Goal", most of you will be stuck at the first Step... I'm always a bit surprised as I usually only need a few Secs to find 1-2-3... sometimes 10 Solutions... :o
[Yeah, well, I guess that's why I'm able to answer most Threads on the Forum then, ah-ah...! 8) ]

But hum, my "Advice": Focus on "Simplicity"..., and only go "cumbersome" (and "Creative"...!) when really needed...! :idea:

>>>

I have an EnhReq btw in the following Thread, for adding some 'EXTRACT' Functionality to the 'EVENT' Mode, see Items '-08-' and '-20-', would be "nice" if you could support it... (+ any other EnhReq(s) I have that you might find "useful", ah-ah...!) :wink:
Last edited by chivracq on Thu Nov 08, 2018 6:36 pm, edited 1 time in total.
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
zirjeo
Posts: 82
Joined: Fri Oct 28, 2016 1:49 am

Re: Using EVENT to extract

Post by zirjeo » Tue Sep 11, 2018 12:37 am

Here is the Url.. https://bsaonline.com/SiteSearch/SiteSe ... 3a%3a7%3d2

Trying to extract 2018 summer tax amount for instance
chivracq
Posts: 10301
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Using EVENT to extract

Post by chivracq » Tue Sep 11, 2018 1:15 am

zirjeo wrote:Here is the Url.. https://bsaonline.com/SiteSearch/SiteSe ... 3a%3a7%3d2

Trying to extract 2018 summer tax amount for instance
Okay..., I finally found 8 Matches on "summer tax" on the 'Tax Information' Tab..., but nothing about "2018 summer tax amount" anywhere (would help also if you could give some more precise Info), oh...!, and only 1 Match on "2018 summer tax", getting "closer"...!
General Information for 2018 Summer Taxes
School District 50230 PRE/MBT 100.0000%
Taxable Value $59,414 S.E.V. $80,300
Property Class 401 - Assessed Value $80,300

Tax Bill Number 44696 Last Receipt Number 00006072
Last Payment Date 08/14/2018 Number of Payments 1

Base Tax $2,553.50 Base Paid $2,553.50
Admin Fees $25.53 Admin Fees Paid $25.53
Interest Fees $0.00 Interest Fees Paid $0.00
Total Tax & Fees $2,579.03 Total Paid $2,579.03

Renaissance Zone Not Available Mortgage Code Not Available
=> What exactly do you want to extract...!?

But OK, if you want to extract this whole Content, here is one easy Solution for example:

Code: Select all

VERSION BUILD=8820413 RECORDER=FX
TAB T=1
'URL GOTO=https://bsaonline.com/Tax_SiteSearch/LoadTaxContent?SearchFocus=All%20Records&SearchCategory=Address&SearchText=33256%20Stoner%20Dr&uid=121&PageIndex=1&ReferenceKey=10-36-453-006-000&ReferenceType=0&SearchOrigin=0&RecordKeyDisplayString=10-36-453-006-000&RecordKey=10-36-453-006-000&RecordKeyType=0
SET !EXTRACT NULL
TAG POS=1 TYPE=H3 ATTR=TXT:General<SP>Information<SP>for<SP>2018<SP>Summer<SP>Taxes EXTRACT=TXT
TAG POS=R1 TYPE=TABLE ATTR=* EXTRACT=TXT

'TAG POS=1 TYPE=TD ATTR=TXT:General<SP>Information<SP>for<SP>2018<SP>Summer<SP>Taxes* EXTRACT=HTM

PROMPT {{!EXTRACT}}
This will extract as following:

Code: Select all

General Information for 2018 Summer Taxes[EXTRACT]

"School District","50230","","PRE/MBT","100.0000%"
""
"Taxable Value","$59,414","","S.E.V.","$80,300"
""
"Property Class","401 - ","","Assessed Value","$80,300"
""
" "
"Tax Bill Number","44696","","Last Receipt Number","00006072"
""
"Last Payment Date","08/14/2018","","Number of Payments","1"
""
" "
"Base Tax","$2,553.50","","Base Paid","$2,553.50"
""
"Admin Fees","$25.53","","Admin Fees Paid","$25.53"
""
"Interest Fees","$0.00","","Interest Fees Paid","$0.00"
""
"Total Tax & Fees","$2,579.03","","Total Paid","$2,579.03"
""
" "
"Renaissance Zone","Not Available","","Mortgage Code","Not Available"
""
(Tested on iMacros for FF v8.8.2, PM v26.3.3 (=FF47), Win10_x64.)
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
zirjeo
Posts: 82
Joined: Fri Oct 28, 2016 1:49 am

Re: Using EVENT to extract

Post by zirjeo » Tue Sep 11, 2018 2:27 am

Looks like my link doesn't go directly to the correct tab. If you click on the tax information tab toward the top then you scroll down a bit just looking for it to extract the 2018 summer tax amount of $2,579.03

Literally just the "$2,579.03"

The code I originally posted works but the Tag code just seems like its a bit sluggish. I can deal with it but was just curious if there's something quicker. Because I then have to extract the 2017 winter tax after and deal with sluggish again lol It adds up, and on top of it I have to move on to the next property and deal with it again.
chivracq
Posts: 10301
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Using EVENT to extract

Post by chivracq » Tue Sep 11, 2018 2:39 am

zirjeo wrote:Looks like my link doesn't go directly to the correct tab. If you click on the tax information tab toward the top then you scroll down a bit just looking for it to extract the 2018 summer tax amount of $2,579.03

Literally just the "$2,579.03"

The code I originally posted works but the Tag code just seems like its a bit sluggish. I can deal with it but was just curious if there's something quicker. Because I then have to extract the 2017 winter tax after and deal with sluggish again lol It adds up, and on top of it I have to move on to the next property and deal with it again.
Yeah, the URL I used in my Script was going directly to that 'Tax Information' Tab...

But then, yep OK, you are inside a Table already, then "Standard'" Extraction Techniques inside Tables apply, for example:

Code: Select all

VERSION BUILD=8820413 RECORDER=FX
TAB T=1
'URL GOTO=https://bsaonline.com/Tax_SiteSearch/LoadTaxContent?SearchFocus=All%20Records&SearchCategory=Address&SearchText=33256%20Stoner%20Dr&uid=121&PageIndex=1&ReferenceKey=10-36-453-006-000&ReferenceType=0&SearchOrigin=0&RecordKeyDisplayString=10-36-453-006-000&RecordKey=10-36-453-006-000&RecordKeyType=0

TAG POS=1 TYPE=H3 ATTR=TXT:General<SP>Information<SP>for<SP>2018<SP>Summer<SP>Taxes
TAG POS=R1 TYPE=TH ATTR=TXT:Total<SP>Tax<SP>&<SP>Fees
SET !EXTRACT NULL
TAG POS=R1 TYPE=TD ATTR=TXT:$*  EXTRACT=TXT
PROMPT _{{!EXTRACT}}_
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
zirjeo
Posts: 82
Joined: Fri Oct 28, 2016 1:49 am

Re: Using EVENT to extract

Post by zirjeo » Tue Sep 11, 2018 2:48 am

Ok so it sounds like it just is what it is. I noticed your $* I will have to try that out. Thanks for taking a look.
chivracq
Posts: 10301
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Using EVENT to extract

Post by chivracq » Tue Sep 11, 2018 2:59 am

zirjeo wrote:Ok so it sounds like it just is what it is. I noticed your $* I will have to try that out. Thanks for taking a look.
Yeah well, I kept the '$' Sign in it as I guess it's always there, even if the Amount is "$0.00", I tried with it first and it worked, but you can probably even remove it...

Interesting Site anyway..., neatly designed... 8)
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
zirjeo
Posts: 82
Joined: Fri Oct 28, 2016 1:49 am

Re: Using EVENT to extract

Post by zirjeo » Tue Sep 11, 2018 3:06 am

You may think it's neat but I hate it lol. Out of all the county websites for different states that I scrape from the Michigan one drives me nuts, it slows me down. Is what it is though.
chivracq
Posts: 10301
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Using EVENT to extract

Post by chivracq » Tue Sep 11, 2018 3:16 am

zirjeo wrote:You may think it's neat but I hate it lol. Out of all the county websites for different states that I scrape from the Michigan one drives me nuts, it slows me down. Is what it is though.
Hum, OK, I don't know how the "others" look like, but this one looks neat to me, all Data nicely organized in Sections and Tables, all Data easy to locate (and to extract (with iMacros)), loads quickly..., no 20 Trackers and 3rd Party Scripts, looks pretty neat to me... Give me more of that kind of Sites, ah-ah...! 8)
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
Post Reply