Automate a filling form with a massive .csv in a table

Discussions and Tech Support related to website data extraction, screen scraping and data mining using iMacros.

Moderators: Community Moderators, iMacros Moderators

Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the Google search box (at the top of each forum page) to see if a similar problem or question has already been addressed. This will search the entire contents of the forums as well as the iMacros Wiki.
3. We can respond much faster to your posts if you include the following information:

CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST

Answering your own posts (e.g. attempting to "bump" your topic) drops your topic from the list of unanswered threads, so it may actually receive less views.

Automate a filling form with a massive .csv in a table

by saki-saki on Fri Jul 22, 2016 4:01 pm

Hello!
- iMacros Version: 8.9.70.419
- Browser:Firefox: 47.0.1
- OS: Windows 10 Pro

I need your help regarding the performance of this script I have built in iMacros. It's doing sort of what I intend, sort of :D
I want to load some basic information in a couple of text boxes everytime I click on the element (where the loop should be placed)
ctl00_PlaceHolderMain_BandejaEntidades_LstBandejaEntidades_ctrl{{!LOOP}}_LbtnVer.

The issues I'm having are the following:

1. The table where the data is located only displays 10 elements (although I can click on an index below the table that jumps through every 10 elements= This is called TAG POS=1 TYPE=A ATTR=TXT:354) . I want to loop over 3500 items. In addition to that, and to make things worse, the number of elements grow everyday but backwards: every new element starts at "PlaceHolderMain" in 0 so the first element that was loaded months ago will be the last on the list. But let's ignore this for a moment, so say I want to loop from 1 to 3500 elements, but since it only displays 10 elements, I get an error in the loop, because it cannot find the element 11, for example. How can I loop this to ALL the elements?

2. When I put the condition: SET !SINGLESTEP YES the code seems to work fine (assuming I just want to loop 10 elements, which I AM NOT!, because of the problem aforementioned). However, this is extremely tedious, I just want to run this and get over it. But the code get stuck when it encounters the line FRAME F=1. Not a runtime error, not running out of time, it just stops! Any idea why and how it can be solved?

3. The server response is extremely slow, every step when it makes a call to the server takes about 30-40 seconds. I've tried countless times to solve this with the IT people, but it seems that I cannot expect anything much from them :( Ignoring this, how can I make the script run a little bit faster? Specially the part when it fills out the form and then returns to the main table?

4. When the first loop of filling out the form I got the headers of the .csv file, not the second line that I intended. But since SET !DATASOURCE_LINE {{!LOOP}} would loop every line starting from the element 1, this mean I would have to make another loop? So DATASOURCE starts at 2 but the rows on the table will start at 1 (actually, 0)? I'm really confused right now. Any idea how to solve this? It is my understanding that I would need to create a Javascript file? but I have no idea how to do this!

If someone can give me a hand orientating me on this issues, Im more than grateful!
Edit: I change the comments on the code....



Code: Select all
VERSION BUILD=8970419 RECORDER=FX
TAB T=1
SET !LOOP 1
SET !SINGLESTEP YES
SET !DATASOURCE Prueba_Automata_SIPOV_v2.csv
SET !DATASOURCE_COLUMNS 5
SET !DATASOURCE_LINE {{!LOOP}}
'Login is pre-loaded, it should take me to the main table called "FRMBANDEJAENTIDADES"
URL GOTO=https://www.minagricultura.gov.co/sites/sipov/_layouts/15/SHP-13-MinAgricultura-SIPV/SIPV/FrmBandejaEntidades.aspx
'Click en "Ingresar"
'TAG POS=1 TYPE=INPUT:SUBMIT FORM=ID:aspnetForm ATTR=ID:ctl00_PlaceHolderMain_g_4efd4b82_233f_4f69_a43f_afbd14d8c69e_btnIngresar
'Va a la ultima hoja de la tabla (354)
'TAG POS=1 TYPE=A ATTR=TXT:354
'Click on the button to open another table( "Ver"): ctrl{n} should be the Loop that allow me to access to every single row to fill the data
TAG POS=1 TYPE=A ATTR=ID:ctl00_PlaceHolderMain_BandejaEntidades_LstBandejaEntidades_ctrl{{!LOOP}}_LbtnVer
'To open the frame to fill in the data (WHEN I EXECUTE THIS WITHOUT SET !SINGLESTEP IT STOPS THE SCRIPT, NOT AN ERROR. IT JUST STOPS)
FRAME F=1
'Fill in the textbox with information retrieved from CSV
TAG POS=1 TYPE=TEXTAREA FORM=ID:aspnetForm ATTR=ID:ctl00_PlaceHolderMain_FrmFallo__supervisionEntidades_TxtAccionesDerivadasOrden CONTENT={{!COL2}}
'Fill in the textbox with information retrieved from CSV
TAG POS=1 TYPE=INPUT:TEXT FORM=ID:aspnetForm ATTR=ID:ctl00_PlaceHolderMain_FrmFallo__supervisionEntidades_DtFechaGestion_DtFechaGestionDate CONTENT={{!COL3}}
'Fill in the textbox with information retrieved from CSVo
TAG POS=1 TYPE=INPUT:TEXT FORM=ID:aspnetForm ATTR=ID:ctl00_PlaceHolderMain_FrmFallo__supervisionEntidades_TxtDiasTranscurridos CONTENT={{!COL4}}
'Fill in the textbox with information retrieved from CSV
TAG POS=1 TYPE=TEXTAREA FORM=ID:aspnetForm ATTR=ID:ctl00_PlaceHolderMain_FrmFallo__supervisionEntidades_TxtObservaciones CONTENT={{!COL5}}
'Clicks on the button to add the information placed before
TAG POS=1 TYPE=A ATTR=ID:ctl00_PlaceHolderMain_FrmFallo__supervisionEntidades_LbtnAgregar
'¿Regresa a la tabla inicial?
'FRAME F=1
'No sé!
'WAIT SECONDS=45
'TAG POS=4 TYPE=SPAN ATTR=CLASS:s4-clust&&TXT:
saki-saki
 
Posts: 5
Joined: Mon Jul 18, 2016 6:14 pm

Re: Automate a filling form with a massive .csv in a table

by chivracq on Mon Jul 25, 2016 8:14 am

I started to answer your Thread 2 days ago but didn't/don't have the time to finish it (now), the "Current Page Calculation" is still missing and I didn't answer Issues 2- and 3-... Performance Tuning is always achieved by "tuning" all '!TIMEOUT_xxx' Settings... (+ extracting some HTML Element appearing on the Page eventually...)

saki-saki wrote:Hello!
Code: Select all
- iMacros Version: 8.9.70.419
- Browser:Firefox: 47.0.1
- OS: Windows 10 Pro


I need your help regarding the performance of this script I have built in iMacros. It's doing sort of what I intend, sort of :D
I want to load some basic information in a couple of text boxes everytime I click on the element (where the loop should be placed)
ctl00_PlaceHolderMain_BandejaEntidades_LstBandejaEntidades_ctrl{{!LOOP}}_LbtnVer.

The issues I'm having are the following:

1. The table where the data is located only displays 10 elements (although I can click on an index below the table that jumps through every 10 elements= This is called TAG POS=1 TYPE=A ATTR=TXT:354) . I want to loop over 3500 items. In addition to that, and to make things worse, the number of elements grow everyday but backwards: every new element starts at "PlaceHolderMain" in 0 so the first element that was loaded months ago will be the last on the list. But let's ignore this for a moment, so say I want to loop from 1 to 3500 elements, but since it only displays 10 elements, I get an error in the loop, because it cannot find the element 11, for example. How can I loop this to ALL the elements?

2. When I put the condition: SET !SINGLESTEP YES the code seems to work fine (assuming I just want to loop 10 elements, which I AM NOT!, because of the problem aforementioned). However, this is extremely tedious, I just want to run this and get over it. But the code get stuck when it encounters the line FRAME F=1. Not a runtime error, not running out of time, it just stops! Any idea why and how it can be solved?

3. The server response is extremely slow, every step when it makes a call to the server takes about 30-40 seconds. I've tried countless times to solve this with the IT people, but it seems that I cannot expect anything much from them :( Ignoring this, how can I make the script run a little bit faster? Specially the part when it fills out the form and then returns to the main table?

4. When the first loop of filling out the form I got the headers of the .csv file, not the second line that I intended. But since SET !DATASOURCE_LINE {{!LOOP}} would loop every line starting from the element 1, this mean I would have to make another loop? So DATASOURCE starts at 2 but the rows on the table will start at 1 (actually, 0)? I'm really confused right now. Any idea how to solve this? It is my understanding that I would need to create a Javascript file? but I have no idea how to do this!

If someone can give me a hand orientating me on this issues, Im more than grateful!
p.s. please forgive my english, not my native language, and also please ignore my comments on the code in spanish...i was just trying not to get lost! :oops:

Code: Select all
VERSION BUILD=8970419 RECORDER=FX
TAB T=1
SET !LOOP 1
SET !SINGLESTEP YES
SET !DATASOURCE Prueba_Automata_SIPOV_v2.csv
SET !DATASOURCE_COLUMNS 5
SET !DATASOURCE_LINE {{!LOOP}}
'SET !DATASOURCE_LINE {{!LOOP}}
'Ingresar directamente a "FRMBANDEJAENTIDADES" (Debe estar prelogueado en SharePoint para evitar bucles de autenticación!)
URL GOTO=https://www.minagricultura.gov.co/sites/sipov/_layouts/15/SHP-13-MinAgricultura-SIPV/SIPV/FrmBandejaEntidades.aspx
'Click en "Ingresar"
'TAG POS=1 TYPE=INPUT:SUBMIT FORM=ID:aspnetForm ATTR=ID:ctl00_PlaceHolderMain_g_4efd4b82_233f_4f69_a43f_afbd14d8c69e_btnIngresar
'Va a la ultima hoja de la tabla (354)
'TAG POS=1 TYPE=A ATTR=TXT:354
'Click en el botón "Ver": ctrl{n} debería ser el Loop que permita entrar a cada un de los links
TAG POS=1 TYPE=A ATTR=ID:ctl00_PlaceHolderMain_BandejaEntidades_LstBandejaEntidades_ctrl{{!LOOP}}_LbtnVer
'Abre el FRAME 1 que es la información de las sentencias (SE QUEDA QUIETO)
FRAME F=1
'Ingrese texto en la casilla: "Acciones derivadas de la orden"
TAG POS=1 TYPE=TEXTAREA FORM=ID:aspnetForm ATTR=ID:ctl00_PlaceHolderMain_FrmFallo__supervisionEntidades_TxtAccionesDerivadasOrden CONTENT={{!COL2}}
'Ingrese texto en la casilla: "Fecha de Gestión": Formato mm-dd-yyyy
TAG POS=1 TYPE=INPUT:TEXT FORM=ID:aspnetForm ATTR=ID:ctl00_PlaceHolderMain_FrmFallo__supervisionEntidades_DtFechaGestion_DtFechaGestionDate CONTENT={{!COL3}}
'Dias habiles entre la fecha de notificación y la fecha máxima de cumplimiento
TAG POS=1 TYPE=INPUT:TEXT FORM=ID:aspnetForm ATTR=ID:ctl00_PlaceHolderMain_FrmFallo__supervisionEntidades_TxtDiasTranscurridos CONTENT={{!COL4}}
'Observacion ESCENARIO DE PRUEBA
TAG POS=1 TYPE=TEXTAREA FORM=ID:aspnetForm ATTR=ID:ctl00_PlaceHolderMain_FrmFallo__supervisionEntidades_TxtObservaciones CONTENT={{!COL5}}
'Agrega el estado de la orden
TAG POS=1 TYPE=A ATTR=ID:ctl00_PlaceHolderMain_FrmFallo__supervisionEntidades_LbtnCancelar
'¿Regresa a la tabla inicial?
'FRAME F=1
'No sé!
'WAIT SECONDS=45
'TAG POS=4 TYPE=SPAN ATTR=CLASS:s4-clust&&TXT:

Quoting your whole OP in one Quote and I will quote all Qt's one by one again for structured Answers...

Concerning your Script, like I mentioned at the end of your previous Thread, it will still be handy of you could translate your Comments from SP=>ENG, Comments are always useful in some Code but they become very quickly less useful if you have to submit your Code (for Review or Share) to sbd who doesn't speak the Language, like on this Forum for example...

And, hum..., I tried to have a look at your Site but it is behind Login & Password, so I wasn't able to access it.

But OK, interesting Scenario, and several Issues indeed, but I think I already have a fairly good idea of what you are doing and want to do...

>>>

1- Issue 4-:
4. When the first loop of filling out the form I got the headers of the .csv file, not the second line that I intended. But since SET !DATASOURCE_LINE {{!LOOP}} would loop every line starting from the element 1, this mean I would have to make another loop? So DATASOURCE starts at 2 but the rows on the table will start at 1 (actually, 0)? I'm really confused right now. Any idea how to solve this? It is my understanding that I would need to create a Javascript file? but I have no idea how to do this!

Starting with your 'Issue 4-' as 'Issue 1-' has a Dependency on it, ah-ah...!
(And don't worry, you don't need any JavaScript, well no '.js' Script, even if it could be done using a '.js' Script as well...)

Yep, if your '.CSV' DataSource has a Header in it, =Row_1, you want to start looping your DataSource from Row_2 and you then use:
Code: Select all
SET !LOOP 2
to tell iMacros to stat looping your DataSource from Row_2 and to skip Row_1. And for Page_1 (= Page_355 actually), iMacros will use Rows_[2-11] for the first 10 Items, then Rows_[12-21] for Page_2 (= Page_354), etc...

You will reuse this 'SET !LOOP' Statement probably several times by editing its Value manually in your Script as you will probably run your Script in several Bunches of let's say 1000 Loops (that could run concurrently btw, in 4 different FF Instances or 4 different FF Profiles even, if the Site uses Cookies to ensure there is no messing up with the Cookies, to speed up the whole Processing by about 4 times already) or if your Script for any reason stops at Loop=134 for example. Then you edit your Macro to:
Code: Select all
SET !LOOP 134
... your Script will start again from Loop=134 (= Row_134 in the DataSource = Item_Nb=133.

For splitting the whole Processing in 4 Batches,
Macro_1_[1-1000] will use:
Code: Select all
SET !LOOP 2
... and will have Loop_Max=1001.

Macro_2_[1001-2000] will use:
Code: Select all
SET !LOOP 1002
... and will have Loop_Max=2001.

Macro_3_[2001-3000] will use:
Code: Select all
SET !LOOP 2002
... and will have Loop_Max=3001.

Macro_4_[3001-3550] will use:
Code: Select all
SET !LOOP 3002
... and will have Loop_Max=3560.

As you use '{{!LOOP}}' as well for a TAG Statement in
Code: Select all
TAG POS=1 TYPE=A ATTR=ID:ctl00_PlaceHolderMain_BandejaEntidades_LstBandejaEntidades_ctrl{{!LOOP}}_LbtnVer
... which means that you will need to use/compute your own Loop, based on '!LOOP' like in:
Code: Select all
SET Item_Nb {{!LOOP}}
ADD Item_Nb -1
TAG POS=1 TYPE=A ATTR=ID:ctl00_PlaceHolderMain_BandejaEntidades_LstBandejaEntidades_ctrl{{Item_Nb}}_LbtnVer


>

2- Issue 1-:
1. The table where the data is located only displays 10 elements (although I can click on an index below the table that jumps through every 10 elements= This is called TAG POS=1 TYPE=A ATTR=TXT:354) . I want to loop over 3500 items. In addition to that, and to make things worse, the number of elements grow everyday but backwards: every new element starts at "PlaceHolderMain" in 0 so the first element that was loaded months ago will be the last on the list. But let's ignore this for a moment, so say I want to loop from 1 to 3500 elements, but since it only displays 10 elements, I get an error in the loop, because it cannot find the element 11, for example. How can I loop this to ALL the elements?

Yep, so Page_1 is actually Page_355, Page_2 = Page_354, etc...
Here you will need "Nested Loops", 2 actually. But they are both at the same "Level" of Nesting, the 2nd one is not nested again in the 1st nested one, and you can even use only one Nested Loop to reuse for the other one. Ah-ah...!, interesting Concept, hold on, I'll explain, it's very simple Maths...!
We've already handled your Loop starting at 2because of the Header in your 'CSV', so when talking about "Loop", I mean now your 'Item_Nb' Loop.
Every 10 Loops, you will need to click on the Page_Nb (=354) either at the end of Loop=10 or at the beginning of Loop=11, =353 for Loop=20/21 etc...
You will first define a 'Page' Variable:
Code: Select all
SET Pages_Total 355
SET Page_Current {{!Page_Total}}

The "aim of the game" now is, using 'EVAL()' to compute a 0 / 1 to subtract to 'Page_Current' every 10 Loops, for Loops =21, =31, etc and to only click on the Page Link on those Loops.

It could be done in just one 'EVAL()' Statement, but I'm too lazy and I just reuse some Code I use for myself in one of my own Scripts...:
Code: Select all
SET Loop_Modulo10 EVAL ("var d='{{Item_Nb}}'; var x; var y=Math.ceil(d/10); x=(d-(y-1)*10); x;")
SET Next_Page EVAL("var d='{{Loop_Modulo10}}'; var x; if(d==10){x=1;} else {x=0;}; x;")

And you will reuse this 1/0 for 2 things; Decrement 'Page_Current' every 10 Loops and to click on the Page Link only on every 10th Loop.
Ah!, but grrr...!, I cannot be lazy, I still need to calculate the Current Page Nb, it can probably be extracted from the Page itself or maybe from the URL, but as I don't know how the Page looks like, I need to calculate it...:
Code: Select all
SET !ERRORIGNORE YES
SET Pages_Total 355
SET Page_Current {{!Page_Total}

SET !LOOP 2
SET Item_Nb {{!LOOP}}
ADD Item_Nb -1
TAG POS=1 TYPE=A ATTR=ID:ctl00_PlaceHolderMain_BandejaEntidades_LstBandejaEntidades_ctrl{{Item_Nb}}_LbtnVer

SET Loop_Modulo10 EVAL ("var d='{{Item_Nb}}'; var x; var y=Math.ceil(d/10); x=(d-(y-1)*10); x;")
SET Next_Page EVAL("var d='{{Loop_Modulo10}}'; var x; if(d==10){x=1;} else {x=0;}; x;")
SET Current_Page EVAL("var pt='{{Page_Total}}'; var d='{{Item_Nb}}'; var y=Math.ceil(d/10);
'>
ADD Page_Current -{{Next_Page}}

'TAG POS=1 TYPE=A ATTR=TXT:354
TAG POS=1 TYPE=A ATTR=TXT:{{Page_Current}}

'Debug:
PROMPT LOOP:<SP>_{{!LOOP}}_<BR>Item_Nb:<SP>_{{Item_Nb}}_<BR>Loop_Modulo10:<SP>_{{Loop_Modulo10}}_<BR>Next_Page:<SP>_{{Next_Page}}_<BR>Page_Total:<SP>_{{Page_Total}}_<BR>Page_Current:<SP>_{{Page_Current}}_
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all 3 Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
chivracq
 
Posts: 6474
Joined: Sat Apr 13, 2013 6:07 am
Location: Amsterdam (NL)

Re: Automate a filling form with a massive .csv in a table

by saki-saki on Tue Jul 26, 2016 4:40 pm

Hello Chivracq!
Thank you for your quick reply. I'm verry sorry for asking you again this bunch of questions, I know I'm taking for granted your help!.
Maybe I should point out that I'm not a software engineer/developer /IT guy, and my experience with programming is limited, although I'm very keen to learn this. Rather than inputting this manually I would really like to learn how to improve this process.
Anyway, I tried to run the code you provided me before, the last bit ( I slightly modified because of probable typos?).
Let's say I just want to "turn the pages"on this, without doing anything else, like when you're in the dentist with a bunch of old magazines...this should do the trick right? step trough every page number and go the next...
chivracq wrote:Ah!, but grrr...!, I cannot be lazy, I still need to calculate the Current Page Nb, it can probably be extracted from the Page itself or maybe from the URL, but as I don't know how the Page looks like, I need to calculate it...:
Code: Select all
SET !ERRORIGNORE YES
SET Pages_Total 355
SET Page_Current {{!Page_Total}

SET !LOOP 2
SET Item_Nb {{!LOOP}}
ADD Item_Nb -1
TAG POS=1 TYPE=A ATTR=ID:ctl00_PlaceHolderMain_BandejaEntidades_LstBandejaEntidades_ctrl{{Item_Nb}}_LbtnVer

SET Loop_Modulo10 EVAL ("var d='{{Item_Nb}}'; var x; var y=Math.ceil(d/10); x=(d-(y-1)*10); x;")
SET Next_Page EVAL("var d='{{Loop_Modulo10}}'; var x; if(d==10){x=1;} else {x=0;}; x;")
SET Current_Page EVAL("var pt='{{Page_Total}}'; var d='{{Item_Nb}}'; var y=Math.ceil(d/10);
'>
ADD Page_Current -{{Next_Page}}

'TAG POS=1 TYPE=A ATTR=TXT:354
TAG POS=1 TYPE=A ATTR=TXT:{{Page_Current}}

'Debug:
PROMPT LOOP:<SP>_{{!LOOP}}_<BR>Item_Nb:<SP>_{{Item_Nb}}_<BR>Loop_Modulo10:<SP>_{{Loop_Modulo10}}_<BR>Next_Page:<SP>_{{Next_Page}}_<BR>Page_Total:<SP>_{{Page_Total}}_<BR>Page_Current:<SP>_{{Page_Current}}_


The prompt alert is giving me this output:
Code: Select all
LOOP: _1_
Item_Nb: _0_
Loop_Modulo10: _10_
Next_Page: _1_
Page_Total: _355_
Page_Current: _

So, I'm noticing that the variable: Page_Current is (maybe) blank? In any case it should provide me the page #1 if I'm running the first loop, isn't it?
When I set the LOOP starting from 1, I can get into the table that I want to input data. However, when I set it from 11, say, nothing happens. I mean, this table does not show up.

Since I'm afraid to mess up everything, I just tried to extract some data with the loop structure that you provided me. Let's say I want to extract a textbox from the element "11" to "21" of one element, coded like this:

Code: Select all
TAG POS=1 TYPE=INPUT:TEXT FORM=* ATTR=ID:ctl00_PlaceHolderMain_FrmFallo_TxtCedulaCatastral EXTRACT=TXT
PROMPT {{!EXTRACT}}
SAVEAS TYPE=EXTRACT FOLDER=* FILE=WHAT.csv


The code seems to loop fine, However, it just throws #EANF#. I've tried with different elements, same history. Weird thing is, when I do this with items 1 from 10, it prompts the actual elements on the table.
But after the element 11, nothing happens, just #EANF#.
My ignorant guess is that it is not calculating correctly the current page...as you said that without knowing how the page works is very hard to understand the root of the cause.

OK, so I tried a different aproach (albeit totally naif):
I've noticed two things:
- Theres a drop-down list that allows me to show 10,50 or 100 elements: This is tagged:
Code: Select all
TAG POS=1 TYPE=SELECT FORM=ID:aspnetForm ATTR=NAME:example_length&&ARIA-CONTROLS:example&&CLASS: CONTENT=%100


- The "index" of pages obviously will change (from 354 to 36 pages).

This is tagged (PAGE 2):
Code: Select all
TAG POS=1 TYPE=A ATTR=CLASS:paginate_button<SP>&&ARIA-CONTROLS:example&&DATA-DT-IDX:2&&TABINDEX:0


Lazy as I am, i just copy/pasted the last bit of code from 1 to 36, so it can turn all this pages one by one.
However, I realized that the attribute DATA-DT-IDX just goes from 1 to 5. So, I cannot loop this through the 36 pages. It just goes to 1 to 5 over and over. Right when I'm in the page N°5, it starts over to 1. ( It actually gives me an error saying that DATA-DT-IDX after 5 does not exist).
This make no sense to me, but anyhoo :wink:

So, chivracq, what I'm doing wrong in this case? I can feel that I'm getting to the goal, but with my eyes closed...I've seen your replies to other topics, and I know you're the MAN! Is it possible that I share with you the complete page (html?) so you can take a look inside?
If not, just shed me some light in here... will be highly appreciated!

This is my code so far:
Code: Select all
VERSION BUILD=8970419 RECORDER=FX
TAB T=1
URL GOTO=https://www.minagricultura.gov.co/sites/sipov/_layouts/15/SHP-13-MinAgricultura-SIPV/SIPV/FrmBandejaEntidades.aspx
'SET !SINGLESTEP YES
SET !ERRORIGNORE YES
SET Page_Total 355
SET Page_Current {{!Page_Total}}


SET !LOOP 11
SET Item_Nb {{!LOOP}}
ADD Item_Nb -1
'To evaluate current page of the loop
SET Loop_Modulo10 EVAL ("var d='{{Item_Nb}}'; var x; var y=Math.ceil(d/10); x=(d-(y-1)*10); x;")
SET Next_Page EVAL("var d='{{Loop_Modulo10}}'; var x; if(d==10){x=1;} else {x=0;}; x;")
SET Current_Page EVAL("var pt='{{Pages_Total}}'; var d='{{Item_Nb}}'; var y=Math.ceil(d/10);")
ADD Page_Current -{{Next_Page}}

'This should click the tab index of pages
'TAG POS=1 TYPE=A ATTR=CLASS:DATA-DT-IDX:{{Page_Current}}
'Click on the link to open the form
TAG POS=1 TYPE=A ATTR=ID:ctl00_PlaceHolderMain_BandejaEntidades_LstBandejaEntidades_ctrl{{Item_Nb}}_LbtnVer
'FRAME F=1
'WAIT SECONDS=15
'Extract some element
TAG POS=1 TYPE=INPUT:TEXT FORM=* ATTR=ID:ctl00_PlaceHolderMain_FrmFallo_TxtCedulaCatastral EXTRACT=TXT
PROMPT {{!EXTRACT}}
SAVEAS TYPE=EXTRACT FOLDER=* FILE=WHAT.csv


'Debug:
PROMPT LOOP:<SP>_{{!LOOP}}_<BR>Item_Nb:<SP>_{{Item_Nb}}_<BR>Loop_Modulo10:<SP>_{{Loop_Modulo10}}_<BR>Next_Page:<SP>_{{Next_Page}}_<BR>Page_Total:<SP>_{{Page_Total}}_<BR>Page_Current:<SP>_{{Current_Page}}_
Attachments
Barra.png
Barra.png (2.67 KIB) Viewed 968 times
saki-saki
 
Posts: 5
Joined: Mon Jul 18, 2016 6:14 pm

Re: Automate a filling form with a massive .csv in a table

by chivracq on Tue Jul 26, 2016 6:25 pm

Oh...!, I notice I made a double-typo in:
Code: Select all
SET Pages_Total 355
SET Page_Current {{!Page_Total}}

=> "Pages_Total" <> "!Page_Total". You need to use the exact same Name for the Variable, let's call it "Page_Total", like I later used it in the 'EVAL()' Statement and in the PROMPT... =>
Code: Select all
SET Page_Total 355
SET Page_Current {{Page_Total}}

About 10-50-100 Results per Page, yep, it's (normally) always better (Performance-wise) to have the Max-Nb of Results per Page, unless if that Page needs to reload again for each Loop, then it will be quicker with only 10 Results per Page. For the 'Loop_Modulo[n]' Formula I gave, the Value 10 can easily be changed (my original own Script used "20"), it could be defined as a Var at the beginning of the Script for easy Access, and could even automatically be retrieved from the Page.

But yep, if I could see how the Site/Pages look like (HTML-Structure-wise) and play with the Pages myself, that would make things easier because currently I'm kind of trying to help you like trying to deactivate a Bomb blindfolded over the Phone...! Direct Access to the Site would be best, or if you can upload a few Pages (zipped, Max 256Kb) to your Thread, or you can send them per Mail to me if you don't want to upload them to the Forum @ '[My_Pseudo_on_the_Forum]@gmail.com'.
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all 3 Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
chivracq
 
Posts: 6474
Joined: Sat Apr 13, 2013 6:07 am
Location: Amsterdam (NL)


Return to Data Extraction and Web Screen Scraping

Who is online

Users browsing this forum: Majestic-12 [Bot] and 2 guests

-->