Topic: Problem with invalid character

I am trying to import some data into another system from a Recorder 6 report, which is being rejected because of an illegal character. I suspect the problem is that one of the numerous records I have imported from xls, xlsx and csv files has contained an invalid character, most likely in a comment field. The issue is trying to locate the problem to correct it.

The steps I have followed are:
1) Run report in Recorder 6
2) Export as an Excel file
3) Perform some trivial edits on the Excel file (e.g. change " Count" to "" for abundance data)
4) Save data as CSV file
5) Import into other system, assuming import file is encoded WIN1252, which it then converts to UTF-8
6) Illegal character 0x081 rejects the import.
The problem is that the line number reported for an illegal character does not have any problems, so assume line numbers have gone astray because of a problem earlier or later. The issue is that the files contains over 800,000 records.

What character encoding does Recorder 6 assume when importing data?
What character encoding does SQL Server 2014 use to store Recorder 6 data?
What character encoding does Recorder 6 export its data?
How can I find where the problem is to correct it?

Harry Clarke
Surrey County Butterfly Recorder


Re: Problem with invalid character

I've eventually managed to track down the problem to some strange characters that had been input into the Location Name. Whilst the problem is resolved. It doesn't explain how illegal characters where allowed to be imported in the first place.

Harry Clarke
Surrey County Butterfly Recorder