proper filling of csv columns

Discussions and Tech Support related to website data extraction, screen scraping and data mining using iMacros.
Forum rules
iMacros EOL - Attention!

The renewal maintenance has officially ended for Progress iMacros effective November 20, 2023 and all versions of iMacros are now considered EOL (End-of-Life). The iMacros products will no longer be supported by Progress (aside from customer license issues), and these forums will also no longer be moderated from the Progress side.

Thank you again for your business and support.

Sincerely,
The Progress Team

Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information: CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
Post Reply
STRONGpassword
Posts: 7
Joined: Wed Jan 02, 2019 10:53 pm

proper filling of csv columns

Post by STRONGpassword » Sun Jan 06, 2019 6:59 pm

iMacros=8.9.7, FF=52.xxx OS=WIN7

Code: Select all

FILTER TYPE=IMAGES STATUS=ON
TAB T=1
SET !EXTRACT_TEST_POPUP NO
'SET !ERRORIGNORE YES
SET !TIMEOUT_STEP 0
TAG POS=1 TYPE=B ATTR=TXT:Видео: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R1 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR0 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#1: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R1 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR1 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#1: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R2 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR2 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#1: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R4 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR3 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#2: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R1 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR4 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#2: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R2 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR5 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#2: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R4 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR6 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#3: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R1 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR7 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#3: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R2 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR8 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#3: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R4 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR9 {{!EXTRACT}}
SET !EXTRACT NULL
ADD !EXTRACT {{!var0}}
ADD !EXTRACT {{!var1}}
ADD !EXTRACT {{!var2}}
ADD !EXTRACT {{!var3}}
ADD !EXTRACT {{!var4}}
ADD !EXTRACT {{!var5}}
ADD !EXTRACT {{!var6}}
ADD !EXTRACT {{!var7}}
ADD !EXTRACT {{!var8}}
ADD !EXTRACT {{!var9}}
SAVEAS TYPE=EXTRACT FOLDER=C:\Users\1\Documents\iMacros\Datasources FILE=EXTRACT_2.csv
URL GOTO=imacros://run/?m=EXTRACT_3.iim
issue is next: for example column 2 should contain extracted data !var2, column 3 should contain extracted data !var3
BUT if TAG POS=R2 TYPE=SPAN ATTR=* EXTRACT=TXT is missed (doesn't exist) it automatically replaced by the next !var
Let me be more clear: column1 = surname, column2 = name, column3 = city etc
but if name is missed, column2 will contain city.
How to solve it, how to mark non existing element with proper signature
and get next column1 = surname, column2 = missed, column3 = city
chivracq
Posts: 10301
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: proper filling of csv columns

Post by chivracq » Sun Jan 06, 2019 7:52 pm

STRONGpassword wrote:
Sun Jan 06, 2019 6:59 pm

Code: Select all

iMacros=8.9.7, FF=52.xxx OS=WIN7

Code: Select all

FILTER TYPE=IMAGES STATUS=ON
TAB T=1
SET !EXTRACT_TEST_POPUP NO
'SET !ERRORIGNORE YES
SET !TIMEOUT_STEP 0
TAG POS=1 TYPE=B ATTR=TXT:Видео: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R1 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR0 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#1: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R1 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR1 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#1: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R2 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR2 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#1: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R4 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR3 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#2: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R1 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR4 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#2: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R2 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR5 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#2: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R4 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR6 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#3: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R1 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR7 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#3: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R2 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR8 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#3: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R4 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR9 {{!EXTRACT}}
SET !EXTRACT NULL
ADD !EXTRACT {{!var0}}
ADD !EXTRACT {{!var1}}
ADD !EXTRACT {{!var2}}
ADD !EXTRACT {{!var3}}
ADD !EXTRACT {{!var4}}
ADD !EXTRACT {{!var5}}
ADD !EXTRACT {{!var6}}
ADD !EXTRACT {{!var7}}
ADD !EXTRACT {{!var8}}
ADD !EXTRACT {{!var9}}
SAVEAS TYPE=EXTRACT FOLDER=C:\Users\1\Documents\iMacros\Datasources FILE=EXTRACT_2.csv
URL GOTO=imacros://run/?m=EXTRACT_3.iim
issue is next: for example column 2 should contain extracted data !var2, column 3 should contain extracted data !var3
BUT if TAG POS=R2 TYPE=SPAN ATTR=* EXTRACT=TXT is missed (doesn't exist) it automatically replaced by the next !var
Let me be more clear: column1 = surname, column2 = name, column3 = city etc
but if name is missed, column2 will contain city.
How to solve it, how to mark non existing element with proper signature
and get next column1 = surname, column2 = missed, column3 = city
It's a Bug actually, or I consider it a Bug, even if it was never acknowledged by TechSup...

When the 1st 'EXTRACT'/Value/Var added to the '!EXTRACT' Var is an empty String, it is simply ignored by the 'EXTRACT' Mechanism, which causes indeed all Extracts to shift by 1 Cell to the Left... :oops:

There are several Workarounds, and in your Case, as you are anyway already kind of re-populating the '!EXTRACT' Var yourself just before doing the 'SAVEAS' instead of letting iMacros automatically add all 'EXTRACT''s itself (justified by the fact that you are using 'R-POS' for all 'EXTRACT''s to avoid re-extracting your Anchor several times, which could actually possibly be avoided if your Anchor is not clickable), would be to go one step even further, and instead of your:

Code: Select all

SET !EXTRACT NULL
ADD !EXTRACT {{!var0}}
ADD !EXTRACT {{!var1}}
ADD !EXTRACT {{!var2}}
ADD !EXTRACT {{!var3}}
ADD !EXTRACT {{!var4}}
ADD !EXTRACT {{!var5}}
ADD !EXTRACT {{!var6}}
ADD !EXTRACT {{!var7}}
ADD !EXTRACT {{!var8}}
ADD !EXTRACT {{!var9}}
... => to replace that part with:

Code: Select all

SET !EXTRACT {{!VAR0}}[EXTRACT]{{!VAR1}}[EXTRACT]...[...]...[EXTRACT]{{!VAR9}}
PROMPT _{{!EXTRACT}}_
With this Workaround, even if '!VAR0' is an empty String, it will still be taken into account and your Cols won't shift to the Left... 8)

>>>

Instead of using the 10x '!VAR[0-9]' Built-in Vars, you could/should rather use 'User Defined Vars' and give them some "meaningful" Names... (And you won't be limited to only 10x Vars...) :idea:

I see in your Script also that you are using the 'FILTER' Command, but hum, that Command got actually broken from FF52 I think, and Pages that are "heavy" on Images actually load quicker with 'FILTER'="OFF" from FF52... :oops:
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
Post Reply