While trying to import a large dataset into my database using Squirrel I encountered the following error.
[Dataimport Executor Thread] ERROR net.sourceforge.squirrel_sql.plugins.dataimport.ImportDataIntoTableExecutor - Error while reading file java.io.IOException: Maximum column length of 100,000 exceeded in column 3 in record 519,064. Set the SafetySwitch property to false if you're expecting column lengths greater than 100,000 characters to avoid this error. at com.csvreader.CsvReader.readRecord(CsvReader.java:841) at net.sourceforge.squirrel_sql.plugins.dataimport.importer.csv.CSVFileImporter.next(CSVFileImporter.java:123) at net.sourceforge.squirrel_sql.plugins.dataimport.ImportDataIntoTableExecutor._execute(ImportDataIntoTableExecutor.java:160) at net.sourceforge.squirrel_sql.plugins.dataimport.ImportDataIntoTableExecutor.access$000(ImportDataIntoTableExecutor.java:51) at net.sourceforge.squirrel_sql.plugins.dataimport.ImportDataIntoTableExecutor$1.run(ImportDataIntoTableExecutor.java:108)
I tried looking for this SafetySwitch property and found out that it was somewhere hidden in the source-code.
I checked out the source code and recompiled the plugin with the SafetySwitch off. Additionally, if there is a SQL exception,the query will be outputted to the logger. I’ve tested the updated plugin in Squirrel-3.4.0-Optional and it worked flawlessly: You can download the jar here (use at own risk).
I spent some time doing it the right way. I’ve added a checkbox to enable or disable the SafetySwitch:
You can download the update jar here.
To install this plugin, simply replace the dataimport.jar in the plugins folder from SquirrelSQL and then restart.
I’ve filed the changes to this plugin as a patch so maybe it’ll get included in the next release 🙂 .
A man can dream 🙂