The complete guide to understanding and using easy populate and understanding CSV file formating and more
This complete guide to useing and understanding easy poulate will teach you everything you need to know about using the easy populate add on/contribution available for and with CartStore, osCommerce, Zen Cart, CRE Loaded and other similar shopping cart software that are based on the oscommerce shopping cart technology which ourr best open source shopping cart's including cartstore have evolved from, if you have any questions please post a comment and I will post back and update the article to include, just in case I forgot to mention anything.
What is easy populate? In short easy populate is a shopping cart import/export software application written in the php programming language. What it does is gather fields that consists of multiple or it could be as well a single product's data that is in the data base that makes up the data or information for product details such as price, inventory stock levels, attributes, product descriptions etc. Easy popualte is capable of importing and exporting the aforementioned data from and to a mysql database and writing the data to a file formatted in the csv format, then a user using software such as Microsoft excel or a free alternative excel clone such as open office calc can open this csv file and make changes to data. Then a user can import this csv file using the easy popualte web interface and quickly and easily updated thousands of products information at a single time, easy popualte can trim days , months or even years in certain circumstances of work down to hours, work that would have typically had to have been done by making changes in multiple areas of a stores interface, time consuming page loads such as loading 2 pages or 3 to make a change to a products data that took no more then a keystroke and automated operation via easy populates csv file import funcitonality.
This article is copyrighted not open source , you can link to this article but do not copy it.Don't let the name confuse you there really is nothing easy about it unless you understand, phpmyadmin, ftp, csv formats, mysql and php escape characters, have some degree of experience in debugging issues with character problems causing import issues and know how to backup the database and restore and have done it b efore and know what you are doing. The only reason we havent renamed it to a more acceptable name like "csv import" is to let users know where the technology comes from so they can search the net, find resources on it , find solutions to problems others encountered while using easy populate etc. etc.
I’m going to lay out a few easy populate rules/guidelines/knowledge here, this knowledge gained from more then 4 years working with easy pop and to the best of my knowledge everything you need to know.
For large database imports you’re going to need most likely your own dedicated server unless you can adjust the php script execution time limit. If you can’t adjust the php execution time limits ten you need to split the csv files into separate files, there is no set rule as to how many rows you have its more or less dependent on the speed of the server and the amount of data in the columns as well as how many fields (columns) you are importing. On a average with a server not adjusted for large execution time expect to be able to import about 1,000 products with average descriptions via easy populate without having to split the file, also note that only our easy populate due to enhancements can import so many products. Ours on a dedicated server with large execution limits can import over 30,000 products at a single time, we have not tried any more then that nor have we hit a execution timeout on our server. Take note if you have a very large database before you do a import you may need to restore a backup the database, best to have root access to the server and run the mysql import command if something goes wrong else you can cause the store to go down for hours. So upload the the sql file forehand uncompressed before importing via easy populate.
For large databases imported via posting via the web easy populate interface if you are using this instead of uploading to a directory you’re going to need large post limits. If you host via WHM/Cpanel both the above limits are easily adjusted through the WHM interface. If your on shared hosting you can contact your hosting company and see if they can either write you a custom php.ini file after you try and fail to write you own and see if it sticks, during this you will find out if your hosting company allows override or there going to need to increase these limits or increase them overall on the server if it’s a small hosting company. Large hosting companies are going to be less flexible and have these limits in place for maximum usage of server resources by hundreds of clients, small hosting companies are going to be more flexible with you and willing to accommodate you. I recommend small hosting companies you can build a relationship with someone there as opposed to large hosting companies such as godaddy which have numerous essential technologies disabled by default, such as low post limits, compression aligrothams such as zip disabled by default. Large hosting companies also tend to have frequent network intrusions, meaning there are hackers in there network as they are targets of achievement for the intruder.
When you import via a csv there are numerous formats of csv you can either have tab, comma delimited csv formatted files etc, etc. We use comma ( , ) delimited by apostrophe ( " ) this is how our easy populate is configured to import in the easy populate php file settings located in the admin folder cartstoreadmin/ in your store's public root directory, you can adjust these settings in the aforementioned file; this is the default csv format type for open office as well as a fairly default csv format for many applications both web and computer based since the beginning of the computing age , it is a very common csv format but also prone to wrong escape character interpretations. Microsoft excel is different in its formating of csv or I should say in its relentless desire to convert your common csv formatted file to its proprietary excel format, if you open the file in excel it and save, it wants to prompt you to save in its propitiatory format which you must say no to, you must keep this format that it originally was in, you must tell it no and you want to keep the format even though certain features may not be available. We have excel safe input technology configured in our cartstore easy populate but generally with easy populate very large imports and numerous delimit characters like that can confuse easy populate with this excel safe import feature. Excel safe import is really only safe with small imports. The more rows you have the more likely easy populate is going to get confused with the escape character and make a wrong decision. This is why tab delimited is so much more optimum.
You can set parameters for easy populate via configuration directives located in teh main easy popualte php file in the admin , to export import via tab delimited format or toher combination of character delimitation's, tabs are a less likely character to appear in any of your product descriptions thus providing a smooth import. You can set easy popualte via configuartion directives to import/export via a tab delimited format and you wont need to clean your product descriptions and other such bad characters or escape them, even old legacy html code such as FONT can cause import issues unless you use tab format or don't clean your descriptions, when you escape a character you place this in front of it / this tells php and mysql that this is not a escape character and not to interpret it as so. We don't have our cartstore easy populate set with tab format as we prefer to clean product descriptions removing old legacy html code and make it more seo friendly for them in the future and more visually appealing and blending uniformally with css. We use the old standard csv import delimitation's as we prefer to carry out our excell/csv work in open office becuase to us its better then excel and it’s free. If you use open office its default csv format type will be set to our stores default type in easy populate which is comma apostrophe delimited.
Easy populate is not easy, we didn't name it. You must make sure you have some degree of technical knowledge in the areas I aforementioned before attempting to use easy populate, if you don't you are going to wipe your database out with numerous errors. You will waste your store's database and may delete your product images trying to clean the mess up. Do not take this litely even if you have successfully imported and exported via easy populate in the past you never know if you have a bad character somewhere that is going to get interpreted as a escape character and throw off your import causing all types of issues. Always , always take a backup via phpmyadmin, the osCommerce database backup interface is ill sufficient for large databases, we are working on finding a solution that can back both database and file system, until then and I always recommend to use phpmyadmin it’s very reliable and you can trust it. We recommended Cpanel based hosting you can take a complete back of the site both database and file and this file can not only be restored to restore your entire hosting account including emails, you can also easily take this to any hosting company that is Cpanel based and they can easily restore your account and emails etc, all of your hosting settings and website. Cpanel in my opinion is by far superior to any hosting control panel out there.
I have just taught you everything you need to know about using easy populate, I have more than 5 years experience using it and have imported exported more than 1,000 stores products, there are many versions of easy populate out there we have a very stable one in our store software, you will find many unstable versions in the oscommerce contrib section of it especially some of the latter ones. Also take note that when importing exporting from store to another store make sure you are using the same version/file of easy populate there many settings such as include tax amount's etc that can cause you a lot of headaches and re imports.
Comments