Join the social network of Tech Nerds, increase skill rank, get work, manage projects...
 
  • Encoding issues while parsing CSV file using Roo gem in rails

    • 0
    • 1
    • 1
    • 0
    • 0
    • 0
    • 0
    • 0
    • 863
    Comment on it

    This is the most common and frustrating issue while parsing a csv file in rails i.e. the encoding issue. So in ruby on rails i found a library which basically detects the Character encoding using ICU (dependency package) and the name of the library is CharlockHolmes. And than we have to add the encoding if it is not already there to parsing it successfully. Below is the code:

    contents = File.read(filepath) # filepath is the path of the file
    begin
        detection = CharlockHolmes::EncodingDetector.detect(contents)   # passing the contents for detection
        encoding  = Encoding.find(detection[:encoding])   # find the encoding from detection object
    rescue => error
        Rails.logger.error("#{error.class}: #{error.message}\n#{error.backtrace.join("\n")}")
        encoding = Encoding::ISO_8859_1   # overwrite if encoding is nil
    end
    
    Roo::CSV.new(filepath, file_warning: :ignore, csv_options: {encoding: encoding})    # adding encoding parameter so that it will not give error
    

    Note: You must install libicu using this command if you are using ubuntu (sudo apt-get install libicu-dev)

 0 Comment(s)

Sign In
                           OR                           
                           OR                           
Register

Sign up using

                           OR                           
Forgot Password
Fill out the form below and instructions to reset your password will be emailed to you:
Reset Password
Fill out the form below and reset your password: