Jason Yates on 10 Mar 2007 00:03:31 -0000


[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]

Re: [PhillyOnRails] importing from csv

  • From: "Jason Yates" <jaywhy@gmail.com>
  • To: talk@phillyonrails.org
  • Subject: Re: [PhillyOnRails] importing from csv
  • Date: Fri, 9 Mar 2007 19:03:10 -0500
  • Dkim-signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=RZG1Ii6KYjeJjYlm2WpQcQAZKMxxdAmNnA2uVYsS3Ugajf1pLpl9XUOfZhcszusKRnn6NfpEehdJsnQaMNqvWHtBvvlF+WIOHGCd2VEaYCN8LYnNBfy8geU1phnyWzMbT4kRHZ5/nToVae6VUlChwKgmJnrk/1eQRrEJL0DSeuE=
  • List-archive: <http://lists.phillyonrails.org/pipermail/talk>
  • Reply-to: talk@phillyonrails.org
  • Sender: talk-bounces@phillyonrails.org
On Mar 7, 2007, at 1:00 PM, Mat Schaffer wrote:
> FasterCSV.foreach(file, :headers => :first_row) do |row|
>       ModelName.create(row.to_hash)
> end

This looks a little more like what I need as a start. 

That'll get you started.  However it will be slow for large amounts of
records.  May even take several minutes.  For speed you may want to
look into MySQL's "LOAD DATA INFILE" statement.  The file must to be
local however.

Also, the "ActiveRecords Extensions" plugin has a import method.
Although not as fast as using the MySQL method above.  The "import"
method allows you to turn validations off and also concatenates
"insert" statements together.  So you don't get punished for opening
and closing the MySQL connection every "insert".  This plugin is MySQL
only, so be aware of that.

You'd use it like this.

ary_of_values = Array.new
FasterCSV.foreach( file ) {|row|  ary_of_values << row.to_a }
Customer.import [:name, :address, :zip, :city, :state, :phone ], ary_of_values

http://www.continuousthinking.com/2007/2/12/activerecord-extensions-0-4-0-released


--
Jason Yates
jaywhy@gmail.com
_______________________________________________
To unsubscribe or change your settings, visit:
http://lists.phillyonrails.org/mailman/listinfo/talk