CSV 1MB filesize limit? [solved]

Discussions and Tech Support specific to the iMacros for Chrome extension.
Forum rules
iMacros EOL - Attention!

The renewal maintenance has officially ended for Progress iMacros effective November 20, 2023 and all versions of iMacros are now considered EOL (End-of-Life). The iMacros products will no longer be supported by Progress (aside from customer license issues), and these forums will also no longer be moderated from the Progress side.

Thank you again for your business and support.

Sincerely,
The Progress Team

Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information: CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
Post Reply
nico.rasters
Posts: 13
Joined: Fri May 15, 2020 11:57 am

CSV 1MB filesize limit? [solved]

Post by nico.rasters » Fri May 15, 2020 12:13 pm

iMacros VERSION BUILD=1005
Chrome Version 81.0.4044.92 (Official Build) (32-bit)
Windows 10 Enterprise (10.0.17134 Build 17134)
Country or region: Netherlands
Windows display language: English (United Kingdom)
I have a Personal Edition license.
The included demo works fine.
I have not tested this in other browsers.
Hope I didn't miss anything. This is my first post.
I could not find any filesize posts on the forum so it seems I'm the only one with this issue. Or it's extremely well known.

Whenever I try to use a csv file that's over 1MB (1015KB in File Explorer still works) I get the error:
Error: Error when communicating with the native messaging host., line: 1

I created a test file by simply entering a bunch of 1s in Excel. I later replaced them with the text "Lorem ipsum" to increase the filesize.
At first I thought there was a limit on the number of rows, then maybe the number of cells so I kept increasing the number of columns as well.
But that's not the case. I can have as many rows and columns as I want, as long as the file stays below 1 MB. It's definitely the filesize. Don't know why, and so far I haven't found a setting to increase it.

iMacros script for testing:
SET !DATASOURCE filename.csv
SET !DATASOURCE_LINE {{!LOOP}}
Last edited by nico.rasters on Wed May 27, 2020 8:54 pm, edited 1 time in total.
chivracq
Posts: 10301
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: CSV 1MB filesize limit?

Post by chivracq » Fri May 15, 2020 12:57 pm

nico.rasters wrote:
Fri May 15, 2020 12:13 pm

Code: Select all

iMacros VERSION BUILD=1005
Chrome Version 81.0.4044.92 (Official Build) (32-bit)
Windows 10 Enterprise (10.0.17134 Build 17134)
Country or region: Netherlands
Windows display language: English (United Kingdom)
I have a Personal Edition license.
The included demo works fine.
I have not tested this in other browsers.

Hope I didn't miss anything. This is my first post.
I could not find any filesize posts on the forum so it seems I'm the only one with this issue. Or it's extremely well known.

Whenever I try to use a csv file that's over 1MB (1015KB in File Explorer still works) I get the error:
Error: Error when communicating with the native messaging host., line: 1

I created a test file by simply entering a bunch of 1s in Excel. I later replaced them with the text "Lorem ipsum" to increase the filesize.
At first I thought there was a limit on the number of rows, then maybe the number of cells so I kept increasing the number of columns as well.
But that's not the case. I can have as many rows and columns as I want, as long as the file stays below 1 MB. It's definitely the filesize. Don't know why, and so far I haven't found a setting to increase it.

iMacros script for testing:

Code: Select all

SET !DATASOURCE filename.csv
SET !DATASOURCE_LINE {{!LOOP}}

Good Quality for a first Post...! :D

Simplified Format to mention your FCI which is also perfect (or at least "good enough for me"):
FCI:

Code: Select all

iMacros for CR v10.0.5 'PE', CR v81.0.4044.92 (Official Build) (32-bit), Windows 10 Enterprise (10.0.17134 Build 17134) Lang=NL/Dutch.
... or even:

Code: Select all

iMacros for CR v10.0.5 'PE', CR81, Win10_1803_x64_Ent (NL/Dutch)
:arrow: Pay attention when you open a Thread to select the "correct" Sub-Forum, the iMacros for FF and CR' Sub-Forum are meant for Threads/Issues that are specific to one of those 2 Browsers, which is usually "incompatible" with your "I have not tested this in other browsers.", ah-ah...! :!:
So you were a bit "lucky" (hum, or unlucky! :wink: ) on this one by opening your Thread in the 'iMacros for CR' Sub-Forum (well, I wouldn't have approved your Thread otherwise... :wink: ), because there is indeed a Size Limitation which is specific to CR only. And it comes from the Browser itself, not from iMacros...

There is a Thread already about "that" on the Forum, but hum..., I don't seem to be able to manage to locate it..., I tried to find it using "chrome+cr+size+limitation" (in any Combination) + @TechSup and myself had posted in it, and it was from "at least" 1 year ago, maybe 2 years ago, but I can't find it back... But I think you are hitting the same Limitation..., coming from CR by Design, I think, and that applies for iMacros for CR to DataSources (and probably for 'SAVEAS TYPE=EXTRACT' also) and even to '.iim' Scripts as Local Files (not Bookmarks)... :oops:

But hum, 1Mb in Size for a '.CSV' or '.TXT' DataSource is very large, even 'Notepad' would take more than 1 Min to open it, it might not be such a "Good Practice" to use such large Files, can't you split or limit its Content a bit, to at least remain below the approx 1Mb Size Limit...? :idea:
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
chivracq
Posts: 10301
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: CSV 1MB filesize limit?

Post by chivracq » Fri May 15, 2020 1:13 pm

Hum..., the Thread I'm trying to find might be older actually, as I see I had mentioned it in this Post, which is already 2 years old... :o

(I will edit this current Post if I manage to locate it...)
=> EDIT...!, as I "think" I managed to find "that Thread" :P , and it was actually from 2015, 5 years ago then, oops...!, and it is a bit long to read..., but still interesting...:
- Limit on number of lines of code
Last edited by chivracq on Fri May 15, 2020 2:06 pm, edited 1 time in total.
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
nico.rasters
Posts: 13
Joined: Fri May 15, 2020 11:57 am

Re: CSV 1MB filesize limit?

Post by nico.rasters » Fri May 15, 2020 1:23 pm

Thank you for the quick reply and the tips regarding post formatting. I had a feeling it was a known issue but couldn't find it.
Everything I'm doing is on the opposite side of good practice. I'm harassing iMacros to automate manual entries into Drupal, where Drupal should actually offer import functionality. And I'm sure it does, but not our implementation.
I'm using Notepad++ to open the csv files, and also to convert them to UTF-8 but that's a different story. I also use Notepad++ to edit the macros because the iMacros editor acts a bit weird around me.
I know there's https://wiki.imacros.net/Database-2-Web.vbs but if I understand correctly scripting is not possible with the Personal Edition (https://imacros.net/store/compare-versions/).
So for now my solution is indeed to split the file into multiple files to stay below the 1MB limit.

And thanks for looking for the original info on the filesize.
chivracq
Posts: 10301
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: CSV 1MB filesize limit?

Post by chivracq » Fri May 15, 2020 3:07 pm

nico.rasters wrote:
Fri May 15, 2020 1:23 pm
Thank you for the quick reply and the tips regarding post formatting. I had a feeling it was a known issue but couldn't find it.
Everything I'm doing is on the opposite side of good practice. I'm harassing iMacros to automate manual entries into Drupal, where Drupal should actually offer import functionality. And I'm sure it does, but not our implementation.
I'm using Notepad++ to open the csv files, and also to convert them to UTF-8 but that's a different story. I also use Notepad++ to edit the macros because the iMacros editor acts a bit weird around me.
I know there's https://wiki.imacros.net/Database-2-Web.vbs but if I understand correctly scripting is not possible with the Personal Edition (https://imacros.net/store/compare-versions/).
So for now my solution is indeed to split the file into multiple files to stay below the 1MB limit.

And thanks for looking for the original info on the filesize.

OK, I "think" I managed to locate the Thread I was looking for, and I've edited my previous Post...

>>>

Yep, using a '.vbs' Script is only supported using the 'Scripting Interface' which is only included in the iMacros Enterprise Edition...

>>>

'Notepad++', OK, very good, I also used it for 2 or 3 years, hum maybe until 8 years ago, when the iMacros Editor (on FF around v6.x/v7.x for FF) was a bit limited and buggy, and I had completely customized and improved the Syntax Highlighting, you have a Thread on the Forum (iMacro script syntax highlighting in Notepad++ editor) to which I might have uploaded my Customizations, I'm not sure anymore, but other (Advanced) Uses had also done it...
Then around 8 years ago, I once had to change and get a new Computer/Laptop, and to reinstall "everything", and before reinstalling 'Notepad++' + adding my Customizations that I needed to retrieve from some external Backup, I gave a try again to the iMacros Editor for FF which had apparently greatly improved "in the meantime", and I've stuck to it since, and it still works "very correctly", at least in the Versions that I use myself: v8.8.2 for FF + v8.9.7 for FF.

I'm not sure about the Editor for v10.0.5 for CR as I don't "really" use that Version, and I only have the 'Free' Version anyway, so I can't test for large Scripts (Limited to 50 Lines) and I can't use '.CSV' Files for a DataSource nor a 'SAVEAS', nor '.iim' Files, but I had done some (Beta) Testing with the Editor from v10.0.2 for FF ('PE'), which I think was based on the Core Code from the Editor from v10.0.1 for CR, which will still be the same for v10.0.5 for CR, and the Editor for v10.0.2 for FF was pretty buggy, ('Find & Replace' for example was broken) and completely unusable with large '.iim' Scripts... I remember opening a 10,000 Line Script would take about 2-3 Min to simply open the Script, + another 2-3 Min to simply resize the Window, which had to happen for every single mini-Edit..., which is of course completely unworkable, pfff...! :shock:

>>>

OK, some "Ideas" for you if you want to try to bypass that Size Limitation for a DataSource...: 8)

- Save your DataSource as a '.TXT' File instead of '.CSV'. (Well, you can simply manually change the Extension.)
Then using the 'file:///' Protocol, you can open that '.TXT' File in CR in a 2nd Tab, if CR lets you open it without "complaining" about the Size Limitation.
The Content of that 'TAB_2' can then be extracted like a "normal" Webpage...
Drawback is that the whole Content of the Page gets extracted in one single HTML (=> 'PRE') Field, and you need to "re-separate" all the Data using 'EVAL()' (+ 'split()'), where hum..., some "other" Size Limitation (for Vars in 'EVAL()') might occur , and in order to be able to re-separate the Data, you also need to separate all Columns with some extra "fake" Cols using "#" and "##" for example to use as Separator...
It works well and quick for "short" Files, but hum..., maybe a bit cumbersome...!

And this Technique doesn't work with '.CSV', because the Browser will not want to open the Local File in a 2nd Tab, but only wants to "download" it... Yep, even if the File is "already" Local, ah-ah...! Hum, at least if the '.CSV' Extension is associated with 'Excel' (or 'OO'/'LO'), I'm not completely sure if that Behaviour happens as well if '.CSV' is associated with 'Notepad(++)'...

- Maybe more usable, is to save your DataSource as '.html' File, I'm not sure anymore if that can be done from 'Notepad++' directly, but 'Excel'/'OO'/'LO' can do it... Then same like with a '.TXT' (Local) File, open it in a 2nd Tab, the Browser shouldn't have any Pb opening a 1+Mb large '.html' File, where the whole Data can be extracted Cell by Cell which are all separate HTML Elements, to extract at the 'TABLE'/'TR'(=Row)/'TD'(=Cell) Level. :idea:

Adding (from 'Excel') some "fake" Colours, nearly invisible but just a tiny bit different for all Cols also allows to use those different Colours as an 'ATTR' Parameter for the 'TAG' Mode to identify/specify specific Cols. :idea:
You can also add a Column Header to your Table containing a unique Identifier for each Row, like for example "Row_1"... "Row_78"..."Row_63222", that can also be used with "Row_{{!LOOP}}", (Reason to use "Row_1" and not "Row_00001"), and using 'Relative Positioning' on that Cell/Row Identifier will allow you to locate/tag precisely any Cell in the whole Table. :idea:
Last edited by chivracq on Fri May 15, 2020 3:18 pm, edited 1 time in total.
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
nico.rasters
Posts: 13
Joined: Fri May 15, 2020 11:57 am

Re: CSV 1MB filesize limit?

Post by nico.rasters » Fri May 15, 2020 3:18 pm

God is a DJ. You are a DJ. Hence you are God? Or at least when it comes to iMacros anyway. The solution to have my csv data as an HTML file is a very nice hack. I'm only a little bit over 1MB so I'd be fine with splitting my file in two, but I'm still going to give it a try just for the fun of it.
chivracq
Posts: 10301
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: CSV 1MB filesize limit?

Post by chivracq » Fri May 15, 2020 3:28 pm

nico.rasters wrote:
Fri May 15, 2020 3:18 pm
God is a DJ. You are a DJ. Hence you are God? Or at least when it comes to iMacros anyway. The solution to have my csv data as an HTML file is a very nice hack. I'm only a little bit over 1MB so I'd be fine with splitting my file in two, but I'm still going to give it a try just for the fun of it.

Yep, ah-ah...! I usually only/mostly play in/around A'dam as a DJ..., you'll have to come a bit "up-north" ah-ah...!, if you ever want to dance on my Beat, ah-ah...! Pity for Covid-19, all Festivals and nice Events where I was supposed to play have been cancelled until Septb 2020..., grrr...!! :(

I've added a last Paragraph to my previous Post while you were already posting your Reply... :P
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
nico.rasters
Posts: 13
Joined: Fri May 15, 2020 11:57 am

Re: CSV 1MB filesize limit?

Post by nico.rasters » Fri May 15, 2020 3:40 pm

Thanks again and sorry to hear your events are cancelled.
I myself am having a lot more work because of covid-19. I'm at a university and we're moving education and exams online, and for all the data entry we need automation because there is no import/export functionality available in any of the software that we use. So I'm happy with that table solution because I'll probably be using iMacros for many more things these coming days/weeks/months.
Enjoy your weekend. At least the weather is great.
chivracq
Posts: 10301
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: CSV 1MB filesize limit?

Post by chivracq » Fri May 15, 2020 4:31 pm

nico.rasters wrote:
Fri May 15, 2020 3:40 pm
Thanks again and sorry to hear your events are cancelled.

I myself am having a lot more work because of covid-19. I'm at a university and we're moving education and exams online, and for all the data entry we need automation because there is no import/export functionality available in any of the software that we use. So I'm happy with that table solution because I'll probably be using iMacros for many more things these coming days/weeks/months.

Enjoy your weekend. At least the weather is great.

Yep, I knew already about "University", ah-ah...! (I always check the IP Location/History/Reputation for New Users, before approving their 1st Post..., and that you were not working from Home, ah-ah...!, hum..., unless connected through "Work"-VPN... :twisted: )

But hum, maybe another "Option" for you..., and that would be to use iMacros for FF v8.9.7 which is much more powerful and functional than the v10.0.5 for CR 'PE' that you are currently using... Maybe a "Pity" for your 'PE' License as v8.9.7 for FF was still completely Free, although that would still allow you to run the "Intensive" Automation in the FF Environment, and still use CR for some "other" (lighter/parallel) Automation or "personal" Browsing... :idea:
And v8.9.7 for FF wouldn't hit the 1Mb Size Limitation either... :P

(And I consider CR myself as "pure Malware" anyway, ah-ah...! I "hope" your IT-Dept has implemented some "extended" GPO to keep Control on that "Joke" of a Software... :| )

But hum, v8.9.7 for FF "only" works until FF56, recommended = FF v55.0.3, which will probably not be "allowed" to use by your IT-Dept, (especially after you've "recently" recovered from some Ransomware "Incident", oops...! (OK, not funny...) :wink: ), but if you are allowed to install "other" (modern and up-to-date) Browsers, this iMacros Version also works on 'Pale Moon' (v28) and 'Basilisk' (v2020), Browsers forked on FF but that didn't adopt the "useless" 'WebExtensions' Architecture... :idea:

Hum, nice Weather indeed, I'm nearly surprised "nobody" among my Friends has come yet to my place to launch some "spontaneous" BBQ/DJ "Party"...! :shock:
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
nico.rasters
Posts: 13
Joined: Fri May 15, 2020 11:57 am

Re: CSV 1MB filesize limit?

Post by nico.rasters » Mon May 18, 2020 3:44 pm

Just as a follow up. This weekend I went for the "split the csv files" solution (deadline, so go with what works and play around later), but today I tried the "table" solution and I wanted to share some experiences so others may avoid a headache.
  1. You can't use a file:// URL. It has to be http:// or https://. In other words, it does not work if you open a file on your local computer in your browser. I only found out after installing the xPath Finder plugin. It also couldn't deal with my local file.
  2. You have to use ATTR with TAG POS, otherwise you get an error (wrong format of TAG command). No attributes? Just use ATTR=*.
There are at least two ways to grab text from a certain cell. Either through TAG POS in combination with name attributes, or with TAG XPATH.
Note that even though my HTML code did not contain a <tbody> tag, xPath think it's there. But of course I could have gone straight for //tr.

All that's left to do is build in a loop, and I probably need https://wiki.imacros.net/!LOOP for that.

Sample HTML file:

Code: Select all

<html>
	<body>
		<p>Lorem ipsum</p>
		<table>
			<tr><td name="Row_0">Header 1</td><td name="Row_0">Header 2</td><td name="Row_0">Header 3</td></tr>
			<tr><td name="Row_1">R1C1</td><td name="Row_1">R1C2</td><td name="Row_1">R1C3</td></tr>
			<tr><td name="Row_2">R2C1</td><td name="Row_2">R2C2</td><td name="Row_2">R2C3</td></tr>
			<tr><td name="Row_3">R3C1</td><td name="Row_3">R3C2</td><td name="Row_3">R3C3</td></tr>
		</table>
	</body>
</html>
Sample iMacros script:

Code: Select all

VERSION BUILD=1005

URL GOTO=https://yourwebsite.dom/chivracq.html

TAG POS=1 TYPE=TD ATTR=NAME:Row_1 EXTRACT=TXT
TAG POS=2 TYPE=TD ATTR=NAME:Row_2 EXTRACT=TXT
TAG POS=3 TYPE=TD ATTR=NAME:Row_3 EXTRACT=TXT

' Does the same as above but with xPath 
TAG XPATH="//table/tbody/tr[2]/td[1]" EXTRACT=TXT
TAG XPATH="//table/tbody/tr[3]/td[2]" EXTRACT=TXT
TAG XPATH="//table/tbody/tr[4]/td[3]" EXTRACT=TXT
chivracq
Posts: 10301
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: CSV 1MB filesize limit?

Post by chivracq » Mon May 18, 2020 6:04 pm

nico.rasters wrote:
Mon May 18, 2020 3:44 pm
Just as a follow up. This weekend I went for the "split the csv files" solution (deadline, so go with what works and play around later), but today I tried the "table" solution and I wanted to share some experiences so others may avoid a headache.
  1. You can't use a file:// URL. It has to be http:// or https://. In other words, it does not work if you open a file on your local computer in your browser. I only found out after installing the xPath Finder plugin. It also couldn't deal with my local file.
  2. You have to use ATTR with TAG POS, otherwise you get an error (wrong format of TAG command). No attributes? Just use ATTR=*.
There are at least two ways to grab text from a certain cell. Either through TAG POS in combination with name attributes, or with TAG XPATH.
Note that even though my HTML code did not contain a <tbody> tag, xPath think it's there. But of course I could have gone straight for //tr.

All that's left to do is build in a loop, and I probably need https://wiki.imacros.net/!LOOP for that.

Sample HTML file:

Code: Select all

<html>
	<body>
		<p>Lorem ipsum</p>
		<table>
			<tr><td name="Row_0">Header 1</td><td name="Row_0">Header 2</td><td name="Row_0">Header 3</td></tr>
			<tr><td name="Row_1">R1C1</td><td name="Row_1">R1C2</td><td name="Row_1">R1C3</td></tr>
			<tr><td name="Row_2">R2C1</td><td name="Row_2">R2C2</td><td name="Row_2">R2C3</td></tr>
			<tr><td name="Row_3">R3C1</td><td name="Row_3">R3C2</td><td name="Row_3">R3C3</td></tr>
		</table>
	</body>
</html>
Sample iMacros script:

Code: Select all

VERSION BUILD=1005

URL GOTO=https://yourwebsite.dom/chivracq.html

TAG POS=1 TYPE=TD ATTR=NAME:Row_1 EXTRACT=TXT
TAG POS=2 TYPE=TD ATTR=NAME:Row_2 EXTRACT=TXT
TAG POS=3 TYPE=TD ATTR=NAME:Row_3 EXTRACT=TXT

' Does the same as above but with xPath 
TAG XPATH="//table/tbody/tr[2]/td[1]" EXTRACT=TXT
TAG XPATH="//table/tbody/tr[3]/td[2]" EXTRACT=TXT
TAG XPATH="//table/tbody/tr[4]/td[3]" EXTRACT=TXT

But hum..., the following is not True...: :o
  1. You can't use a file:// URL. It has to be http:// or https://. In other words, it does not work if you open a file on your local computer in your browser.
I don't encounter any Pb opening a Local '.html' File from 'URL GOTO=file:///...'...!
But it is "file:///" and not "file://", => thus with 3 Forward Slashes, and not 2.

For my mini-Test, I did a 'Save as: .html' from 'LO - Calc' (v6.3.1.2, => 'LibreOffice', 'Calc' = same like 'Excel') from the 'Address.csv' Demo DataSource, and the following Script works fine for me:

Code: Select all

VERSION BUILD=1005
SET !EXTRACT_TEST_POPUP NO
TAB T=1
URL GOTO=file:///D:/TEMP/iMacros/Current/iMacros/Datasources/Address.html
'PROMPT _{{!FOLDER_DATASOURCE}}_
'URL GOTO=file:///{{!FOLDER_DATASOURCE}}/Address.html

TAG POS=1 TYPE=TD ATTR=TXT:FNAME
TAG POS=1 TYPE=TD ATTR=TXT:LNAME
TAG POS=1 TYPE=TD ATTR=TXT:ADDRESS
TAG POS=1 TYPE=TD ATTR=TXT:CITY

TAG POS=1 TYPE=TD ATTR=TXT:Frank
TAG POS=R1 TYPE=TD ATTR=TXT:* EXTRACT=TXT
TAG POS=1 TYPE=TD ATTR=TXT:Noris

TAG POS=1 TYPE=TD ATTR=TXT:Thomas
TAG POS=R1 TYPE=TD ATTR=TXT:* EXTRACT=TXT
TAG POS=1 TYPE=TD ATTR=TXT:Test

PROMPT {{!EXTRACT}}
(Tested on iMacros for CR v10.0.5 'Free', CR76, Win10_x64.)

It even still works if I disable "Allow access to file URLs" in the iMacros General Settings...
I had to use the Full Path to the '.htm' File Hard-Coded in the 'URL GOTO', because I only have the 'Free' Version, but you can probably use the '{{!FOLDER_DATASOURCE}}' Var...
Difference with you will be that I still use CR76 while you are on CR81, but I don't think that would make a Difference...

You may also want to have a Look at the HTML Structure of the File, like 'Calc' created it...:

Code: Select all

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">

<html>
<head>
	
	<meta http-equiv="content-type" content="text/html; charset=windows-1252"/>
	<title></title>
	<meta name="generator" content="LibreOffice 6.3.1.2 (Windows)"/>
	<meta name="created" content="00:00:00"/>
	<meta name="changed" content="00:00:00"/>
	
	<style type="text/css">
		body,div,table,thead,tbody,tfoot,tr,th,td,p { font-family:"Liberation Sans"; font-size:x-small }
		a.comment-indicator:hover + comment { background:#ffd; position:absolute; display:block; border:1px solid black; padding:0.5em;  } 
		a.comment-indicator { background:red; display:inline-block; border:1px solid black; width:0.5em; height:0.5em;  } 
		comment { display:none;  } 
	</style>
	
</head>

<body>
<table cellspacing="0" border="0">
	<colgroup width="60"></colgroup>
	<colgroup width="64"></colgroup>
	<colgroup width="115"></colgroup>
	<colgroup width="73"></colgroup>
	<colgroup width="48"></colgroup>
	<colgroup width="67"></colgroup>
	<colgroup width="93"></colgroup>
	<colgroup width="163"></colgroup>
	<tr>
		<td height="17" align="left">FNAME</td>
		<td align="left"> LNAME</td>
		<td align="left"> ADDRESS</td>
		<td align="left"> CITY</td>
		<td align="left"> ZIP</td>
		<td align="left"> STATE ID</td>
		<td align="left"> COUNTRY ID</td>
		<td align="left"> EMAIL </td>
	</tr>
	<tr>
		<td height="17" align="left">Frank</td>
		<td align="left">Noris</td>
		<td align="left">42 Ipswitch Drive</td>
		<td align="left">San Diego</td>
		<td align="right" sdval="45001" sdnum="2057;">45001</td>
		<td align="left">California</td>
		<td align="left">United States</td>
		<td align="left">fn@abc.com</td>
	</tr>
	<tr>
		<td height="17" align="left">Ann</td>
		<td align="left">Coleman</td>
		<td align="left">PO Box 12345</td>
		<td align="left">Ann Arbor</td>
		<td align="right" sdval="48109" sdnum="2057;">48109</td>
		<td align="left">Michigan</td>
		<td align="left">United States</td>
		<td align="left">annc@testemail.org</td>
	</tr>
	<tr>
		<td height="17" align="left">Thomas</td>
		<td align="left">Test</td>
		<td align="left">1 Bills Drive</td>
		<td align="left">NY</td>
		<td align="right" sdval="14128" sdnum="2057;">14128</td>
		<td align="left">New York</td>
		<td align="left">United States</td>
		<td align="left">tomastest@ipswitch.com</td>
	</tr>
</table>
<!-- ************************************************************************** -->
</body>

</html>
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
nico.rasters
Posts: 13
Joined: Fri May 15, 2020 11:57 am

Re: CSV 1MB filesize limit?

Post by nico.rasters » Mon May 18, 2020 9:39 pm

As always your feedback is much appreciated. I will do some more tests with local files and update my post accordingly. Perhaps my HTML file was too simple and not properly structured. I did experiment with an XML file as well, but with the same results.

Ah hell, who needs sleep...

Opening the local file works fine, also with the triple slashes.
However, I'm getting: "Replaying test.iim" in a green bar at the top. It appears to be stuck on the first TAG POS line. I can Pause or Stop the script so it is still running.
Maybe it's a Chrome thing, maybe it has to do with our network (they installed some things after the so called cyber attack), or maybe it's just me ;)

I'm currently on a VPN so I'll give it a try on my home PC as well. But that really is something for tomorrow.
chivracq
Posts: 10301
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: CSV 1MB filesize limit?

Post by chivracq » Mon May 18, 2020 10:27 pm

nico.rasters wrote:
Mon May 18, 2020 9:39 pm
As always your feedback is much appreciated. I will do some more tests with local files and update my post accordingly. Perhaps my HTML file was too simple and not properly structured. I did experiment with an XML file as well, but with the same results.

Ah hell, who needs sleep...

Opening the local file works fine, also with the triple slashes.
However, I'm getting: "Replaying test.iim" in a green bar at the top. It appears to be stuck on the first TAG POS line. I can Pause or Stop the script so it is still running.
Maybe it's a Chrome thing, maybe it has to do with our network (they installed some things after the so called cyber attack), or maybe it's just me ;)

I'm currently on a VPN so I'll give it a try on my home PC as well. But that really is something for tomorrow.

Hum, '.XML' File is "another Story"... Usually nothing gets recorded, or only the very first Top-Top Node, but it can still be extracted at least at the Top-Level, and then you need 'EVAL()' or 'REGEXP'/'XPATH' to isolate/separate the specific Branch/Node/Data that you want... It's a bit cumbersome and not very straightforward, but it can be done...

With a (Local) '.htm'/'.html' File, it is much more straightforward and it just gets treated like a "Standard" HTML Page from a Website, and iMacros "follows" the HTML Structure and the CSS of the File/Page.

Hum, "It appears to be stuck on the first TAG POS line." + talking about a VPN, but hum..., the mini-Script I posted using 'Address.html' converted from the Demo 'Address.csv' does everything Locally, you don't even need an Internet Connection. Does it work for you...?

>>>

I actually tried to update CR to CR81 in the meantime, but because CR behaves so much like some crappy Malware, I "had" completely locked and blocked it a few months ago at CR76, blocking/deleting all Autostart Registry/Task Scheduler Entries and Services and obscure '.EXE' Files running in the Background and reinstalling and restarting themselves if you try to stop/delete anything, thanks to Local Admin Rights secretly granted to them while simply checking in the Browser if by any chance, there might be some Update available, + I had also blocked "everything" Inbound/Outbound with Firewall Rules except the Core Browser, for... browsing the Internet, ah-ah...!, which is the Functionality I expect from a Browser...! :roll: , that hum..., I didn't manage to update it a bit quickly/easily now without re-spending more than 1h giving it 1 Permission back at the time, ah-ah...! :P
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
Post Reply