Share via

“Encountered unexpected character 'ï'” error serializing JSON

  • Article

When trying to deserialize JSON-encoded data from a file using the DataContractJsonSerializer, you may run into the following error:

System.Runtime.Serialization.SerializationException: There was an error deserializing the object of type
ConsoleApplication1.Person. Encountered unexpected character 'ï'. --->
System.Xml.XmlException: Encountered unexpected character 'ï'.
   at System.Xml.XmlExceptionHelper.ThrowXmlException(XmlDictionaryReader reader
, XmlException exception)
   at System.Runtime.Serialization.Json.XmlJsonReader.ReadAttributes()
   at System.Runtime.Serialization.Json.XmlJsonReader.ReadNonExistentElementName (StringHandleConstStringType elementName)
   at System.Runtime.Serialization.Json.XmlJsonReader.Read()
   at System.Xml.XmlBaseReader.IsStartElement()
   at System.Xml.XmlBaseReader.IsStartElement(XmlDictionaryString localName, XmlDictionaryString namespaceUri)
   at System.Runtime.Serialization.XmlReaderDelegator.IsStartElement(XmlDictionaryString localname, XmlDictionaryString ns)
   at System.Runtime.Serialization.XmlObjectSerializer.IsRootElement(XmlReaderDelegator reader, DataContract contract, XmlDictionaryString name, XmlDictionaryString ns)
   at System.Runtime.Serialization.Json.DataContractJsonSerializer.InternalIsStartObject(XmlReaderDelegator reader)
   at System.Runtime.Serialization.Json.DataContractJsonSerializer.InternalReadObject(XmlReaderDelegator xmlReader, Boolean verifyObjectName)
   at System.Runtime.Serialization.XmlObjectSerializer.InternalReadObject(XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContract Resolver)
   at System.Runtime.Serialization.XmlObjectSerializer.ReadObjectHandleExceptions(XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContractResolver)
   --- End of inner exception stack trace ---
   at System.Runtime.Serialization.XmlObjectSerializer.ReadObjectHandleExceptions(XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContractResolver)
    at System.Runtime.Serialization.Json.DataContractJsonSerializer.ReadObject(XmlDictionaryReader reader)
   at System.Runtime.Serialization.Json.DataContractJsonSerializer.ReadObject(Stream stream)
   at ConsoleApplication1.Program.Main(String[] args) in e:\AzureSamples\JSONSerializer\ConsoleApplication1\ConsoleApplication1\Program.cs

One of the reasons for this could be that the input file that contains the JSON-encoded data is created with a binary encoding or it has a Byte Order Mark(BOM) byte sequence for a binary encoded file.
For e.g. The UTF-8 representation of the BOM is the byte sequence (0xEF,0xBB,0xBF) in the beginning of the file.
Note: You will see this if you created a .JSON file(or a binary file) using visual studio.

To verify this, you can open the file using a binary editor like visual studio. To open the Binary Editor, first choose File | New | File from the main menu, select the file you want to edit, then click on the drop arrow next to the Open button, and choose Open With | Binary Editor.

image

image

image 

One way to solve the issue is to delete the BOM byte sequence (0xEF,0xBB,0xBF) in the beginning of the file.

https://connect.microsoft.com/VisualStudio/feedback/details/356750/datacontractjsonserializer-fails-with-non-ansi-characters has the following which might be relevant:

Please note that DataContractJsonSerializer only supports the following encodings: UTF-8, UTF-16LE and UTF-16BE. For this to work, you need to do something like:     
     
MemoryStream ms = new MemoryStream(Encoding.UTF8.GetBytes(panoramio));
DataContractJsonSerializer dcjs = new DataContractJsonSerializer(typeof(PhotoData));
PhotoData pd = (PhotoData)dcjs.ReadObject(ms);

Comments

  • Anonymous
    July 09, 2014
    Thank you very much! I've been looking for days how to solve this issue and didn't found the answer, until now. Does this happen with ALL of the JSON files created through Visual Studio? It's a bit frustrating that the mistake was with the way the serializer encoded the data, not a mistake by myself (I'm a beginner in programming so that's the common thing). Is there a way to avoid this?
  • Anonymous
    August 20, 2014
    Thanks!  This helped me in a little test harness I'm using.  Very good tip.
  • Anonymous
    November 04, 2014
    The comment has been removed
  • Anonymous
    February 05, 2015
    I had the same problem. I solved it like this. I used free NotePad++ . I used the Encoding tool there and encoded the file using "Encode with UTF-8 without BOM". This one click approach resolved the issue.
  • Anonymous
    May 27, 2015
    Muchisimas gracias!!! fue de mucha ayuda!! :D
  • Anonymous
    May 24, 2016
    The comment has been removed
    • Anonymous
      November 11, 2016
      Thanks for you comments Don. Glad that it helped you.
  • Anonymous
    August 27, 2016
    The comment has been removed