proper filling of csv columns

Discussions and Tech Support related to website data extraction, screen scraping and data mining using iMacros.
Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information:CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
Post Reply
STRONGpassword
Posts: 7
Joined: Wed Jan 02, 2019 10:53 pm

proper filling of csv columns

Post by STRONGpassword » Sun Jan 06, 2019 6:59 pm

iMacros=8.9.7, FF=52.xxx OS=WIN7

Code: Select all

FILTER TYPE=IMAGES STATUS=ON
TAB T=1
SET !EXTRACT_TEST_POPUP NO
'SET !ERRORIGNORE YES
SET !TIMEOUT_STEP 0
TAG POS=1 TYPE=B ATTR=TXT:Видео: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R1 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR0 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#1: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R1 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR1 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#1: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R2 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR2 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#1: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R4 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR3 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#2: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R1 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR4 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#2: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R2 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR5 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#2: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R4 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR6 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#3: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R1 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR7 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#3: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R2 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR8 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#3: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R4 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR9 {{!EXTRACT}}
SET !EXTRACT NULL
ADD !EXTRACT {{!var0}}
ADD !EXTRACT {{!var1}}
ADD !EXTRACT {{!var2}}
ADD !EXTRACT {{!var3}}
ADD !EXTRACT {{!var4}}
ADD !EXTRACT {{!var5}}
ADD !EXTRACT {{!var6}}
ADD !EXTRACT {{!var7}}
ADD !EXTRACT {{!var8}}
ADD !EXTRACT {{!var9}}
SAVEAS TYPE=EXTRACT FOLDER=C:\Users\1\Documents\iMacros\Datasources FILE=EXTRACT_2.csv
URL GOTO=imacros://run/?m=EXTRACT_3.iim
issue is next: for example column 2 should contain extracted data !var2, column 3 should contain extracted data !var3
BUT if TAG POS=R2 TYPE=SPAN ATTR=* EXTRACT=TXT is missed (doesn't exist) it automatically replaced by the next !var
Let me be more clear: column1 = surname, column2 = name, column3 = city etc
but if name is missed, column2 will contain city.
How to solve it, how to mark non existing element with proper signature
and get next column1 = surname, column2 = missed, column3 = city
chivracq
Posts: 8523
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: proper filling of csv columns

Post by chivracq » Sun Jan 06, 2019 7:52 pm

STRONGpassword wrote:
Sun Jan 06, 2019 6:59 pm

Code: Select all

iMacros=8.9.7, FF=52.xxx OS=WIN7

Code: Select all

FILTER TYPE=IMAGES STATUS=ON
TAB T=1
SET !EXTRACT_TEST_POPUP NO
'SET !ERRORIGNORE YES
SET !TIMEOUT_STEP 0
TAG POS=1 TYPE=B ATTR=TXT:Видео: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R1 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR0 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#1: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R1 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR1 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#1: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R2 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR2 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#1: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R4 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR3 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#2: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R1 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR4 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#2: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R2 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR5 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#2: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R4 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR6 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#3: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R1 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR7 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#3: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R2 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR8 {{!EXTRACT}}
TAG POS=1 TYPE=B ATTR=TXT:Аудио#3: EXTRACT=TXT
SET !EXTRACT NULL
TAG POS=R4 TYPE=SPAN ATTR=* EXTRACT=TXT
SET !VAR9 {{!EXTRACT}}
SET !EXTRACT NULL
ADD !EXTRACT {{!var0}}
ADD !EXTRACT {{!var1}}
ADD !EXTRACT {{!var2}}
ADD !EXTRACT {{!var3}}
ADD !EXTRACT {{!var4}}
ADD !EXTRACT {{!var5}}
ADD !EXTRACT {{!var6}}
ADD !EXTRACT {{!var7}}
ADD !EXTRACT {{!var8}}
ADD !EXTRACT {{!var9}}
SAVEAS TYPE=EXTRACT FOLDER=C:\Users\1\Documents\iMacros\Datasources FILE=EXTRACT_2.csv
URL GOTO=imacros://run/?m=EXTRACT_3.iim
issue is next: for example column 2 should contain extracted data !var2, column 3 should contain extracted data !var3
BUT if TAG POS=R2 TYPE=SPAN ATTR=* EXTRACT=TXT is missed (doesn't exist) it automatically replaced by the next !var
Let me be more clear: column1 = surname, column2 = name, column3 = city etc
but if name is missed, column2 will contain city.
How to solve it, how to mark non existing element with proper signature
and get next column1 = surname, column2 = missed, column3 = city
It's a Bug actually, or I consider it a Bug, even if it was never acknowledged by TechSup...

When the 1st 'EXTRACT'/Value/Var added to the '!EXTRACT' Var is an empty String, it is simply ignored by the 'EXTRACT' Mechanism, which causes indeed all Extracts to shift by 1 Cell to the Left... :oops:

There are several Workarounds, and in your Case, as you are anyway already kind of re-populating the '!EXTRACT' Var yourself just before doing the 'SAVEAS' instead of letting iMacros automatically add all 'EXTRACT''s itself (justified by the fact that you are using 'R-POS' for all 'EXTRACT''s to avoid re-extracting your Anchor several times, which could actually possibly be avoided if your Anchor is not clickable), would be to go one step even further, and instead of your:

Code: Select all

SET !EXTRACT NULL
ADD !EXTRACT {{!var0}}
ADD !EXTRACT {{!var1}}
ADD !EXTRACT {{!var2}}
ADD !EXTRACT {{!var3}}
ADD !EXTRACT {{!var4}}
ADD !EXTRACT {{!var5}}
ADD !EXTRACT {{!var6}}
ADD !EXTRACT {{!var7}}
ADD !EXTRACT {{!var8}}
ADD !EXTRACT {{!var9}}
... => to replace that part with:

Code: Select all

SET !EXTRACT {{!VAR0}}[EXTRACT]{{!VAR1}}[EXTRACT]...[...]...[EXTRACT]{{!VAR9}}
PROMPT _{{!EXTRACT}}_
With this Workaround, even if '!VAR0' is an empty String, it will still be taken into account and your Cols won't shift to the Left... 8)

>>>

Instead of using the 10x '!VAR[0-9]' Built-in Vars, you could/should rather use 'User Defined Vars' and give them some "meaningful" Names... (And you won't be limited to only 10x Vars...) :idea:

I see in your Script also that you are using the 'FILTER' Command, but hum, that Command got actually broken from FF52 I think, and Pages that are "heavy" on Images actually load quicker with 'FILTER'="OFF" from FF52... :oops:
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all 3 Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
Post Reply