Extracting Product Info,Pics,Price to local MSAccessDB

Discussions and Tech Support related to website data extraction, screen scraping and data mining using iMacros.

Moderators: Community Moderators, iMacros Moderators

Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the Google search box (at the top of each forum page) to see if a similar problem or question has already been addressed. This will search the entire contents of the forums as well as the iMacros Wiki.
3. We can respond much faster to your posts if you include the following information:

CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST

Answering your own posts (e.g. attempting to "bump" your topic) drops your topic from the list of unanswered threads, so it may actually receive less views.

Extracting Product Info,Pics,Price to local MSAccessDB

by Ratfink on Tue Jan 19, 2016 12:27 am

Hi All,
Apologies up front as I am new to iMacros, and Web Data Mining in general.
I'm sure this has been asked before however I did spend a few hours researching and was not able to come up with results.
My particular issue is I have a customer who has a very old ecommerce site. They don't have the code for it. Dev has been out of touch for many years and I have been tasked to redo the site. Problem is no code, no DB for product, pricing, pics, etc.... I essentially need to get all the product info with pics of their old site, get it in a database format, then build new site and populate.

    Need to scrape a website that sells truck parts.
    Need to extract all product info, graphics, prices to a local MS Access DB

Are there any good iMacro examples?
I will continue to dig further, any help would be appreciated.

Thanks
R
Ratfink
 
Posts: 5
Joined: Mon Jan 18, 2016 11:52 pm

Re: Extracting Product Info,Pics,Price to local MSAccessDB

by chivracq on Wed Feb 03, 2016 2:53 pm

Ratfink wrote:Hi All,
Apologies up front as I am new to iMacros, and Web Data Mining in general.
I'm sure this has been asked before however I did spend a few hours researching and was not able to come up with results.
My particular issue is I have a customer who has a very old ecommerce site. They don't have the code for it. Dev has been out of touch for many years and I have been tasked to redo the site. Problem is no code, no DB for product, pricing, pics, etc.... I essentially need to get all the product info with pics of their old site, get it in a database format, then build new site and populate.

    Need to scrape a website that sells truck parts.
    Need to extract all product info, graphics, prices to a local MS Access DB

Are there any good iMacro examples?
I will continue to dig further, any help would be appreciated.

Thanks
R

- Functionality question
Ratfink wrote:Hi all,
This is going to be a newie question. Appolgies up front.
I am not understanding the functionality of iMacros. I think I have figured it out however I need someone to understand my way of thinking to find out and understand its use. Right now I cant seem to get iMacros to do what I want it to do. I have spent weeks reveiwing data extraction examples, instructions, forums, to no avail. I am sure its due to my understanding or lack thereof how it works?

For example, I need to scrape a website that sells car parts. There are over two thousand car parts. I can go to the main page and it shows all the product car parts on the website. 50 products to a page with description, prices, images, title, part number, etc...

What i need to do is to scrape this website for that individual information for all two thousand products.

Here is where I get a bit lost.

How does iMacros function to get this specific information?

1. Recording a macro/ Do I click to the first page of the site, then click to the first product title, then part number, then price, then image,.... Then create some sort of loop that goes to the next product, then aquires the specific info as above, Then to the next pages after collecting 50 products per page? ALL Automatically?

OR

2. Again recording a macro/ Do I have to click to the main web page, click on every single product, all two thousand manually, each one, every page, and record the entire event?

I hope Im not making it complicated, But im trying to figure out how iMacros actaully works? If I can go to the main page that has thousands of product info i need on multiple pages do I need to click thorughj each and everyone on and record a macro. OR do I go to just one product and tell iMacro or iMacro is smart enough to know to loop? to the next product, then the next , then the next and so on?

Im not so sure how much time iMacro is going to save me if I have to record a macro and click on all two thousand individual text tile, description, images, priceing, ext..... I hope someone sees my delimna and can provide an answer.

Hum..., pity you didn't read the Forum Rules when you opened this 1st Thread for Required Info to include in your OP when open a Thread, you've already lost 2 weeks... :roll:
=> CIM...! :mrgreen:
=> Read my Sig, I only read and answer Threads when Users mention their FCI and I see an URL and a Script showing what they've tried a bit and where they get stuck..., so I only read your Question divided into 2 Threads (no "s" for "Question", your both Threads are about the same Scenario) today after you've started your 2nd Thread, you should have bumped this one a bit earlier and I would have reacted about FCI Missing etc..., it's better to keep all Info in one Thread, I think...

OK, very interesting Case btw, mention your FCI and I will follow up..., always willing to help Professionals, but this is a Technical Forum, so you have to play the "Game" a bit in a Technical way...

And post the URL of the Site as well of course, this should have been included in both OP's for your 2 Threads..., and maybe your Script, to see how far you've come..., I won't be writing your whole Script for you... (And I guess you can delete the 2nd Thread, it's better to keep all Info/Answers about a same Scenario/Case in one Thread, unless you stand by your choice for 2 apart Threads..., that could be Legit, but I will then first wait to answer Thread Nb_2 until Thread Nb_1 has been followed up and neatly finished...)
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all 3 Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
chivracq
 
Posts: 6479
Joined: Sat Apr 13, 2013 6:07 am
Location: Amsterdam (NL)

Re: Extracting Product Info,Pics,Price to local MSAccessDB

by Ratfink on Thu Feb 25, 2016 12:36 pm

Your right,
I need to spend the time and read the rules further.....
I will post the neccesary info, and move forward from there.
thanks for info....
Ratfink
 
Posts: 5
Joined: Mon Jan 18, 2016 11:52 pm


Return to Data Extraction and Web Screen Scraping

Who is online

Users browsing this forum: No registered users and 3 guests

-->