We are using HJson to parse regular Json files and found an issue with control characters in Strings.
The input file looks like
{ "aaa" : "xxx\u000byyy" }
and we parse it with HJson. When the resulting JsonValue object is written using toString() or toString(Stringify.FORMATTED) the output contains invalid control characters. It is expected that these control characters are quoted using \u again according to Json standard.
This example program shows the problem:
package de.softquadrat.jdbc.json;
import java.nio.charset.StandardCharsets;
import org.hjson.JsonValue;
import org.hjson.Stringify;
public class Test {
public static void main(String[] args) {
JsonValue hjson = JsonValue.readHjson("{ \"aaa\" : \"xxx\\u000byyy\" }");
String s = hjson.toString(Stringify.FORMATTED);
System.out.println(hexdump(s));
JsonValue.readHjson(s);
}
// from https://gist.github.com/geekman/5243537
public static String hexdump(String s) {
byte[] data = s.getBytes(StandardCharsets.UTF_8);
final int perRow = 16;
final String hexChars = "0123456789ABCDEF";
StringBuilder dump = new StringBuilder();
StringBuilder chars = null;
for (int i = 0; i < data.length; i++) {
int offset = i % perRow;
if (offset == 0) {
chars = new StringBuilder();
dump.append(String.format("%04x", i))
.append(" ");
}
int b = data[i] & 0xFF;
dump.append(hexChars.charAt(b >>> 4))
.append(hexChars.charAt(b & 0xF))
.append(' ');
chars.append((char) ((b >= ' ' && b <= '~') ? b : '.'));
if (i == data.length - 1 || offset == perRow - 1) {
for (int j = perRow - offset - 1; j > 0; j--)
dump.append("-- ");
dump.append(" ")
.append(chars)
.append('\n');
}
}
return dump.toString();
}
}
The output of the test program gives:
0000 0A 7B 0A 20 20 22 61 61 61 22 3A 20 22 78 78 78 .{. "aaa": "xxx
0010 0B 79 79 79 22 0A 7D -- -- -- -- -- -- -- -- -- .yyy".}
Exception in thread "main" org.hjson.ParseException: Expected valid string character at 3:13
at org.hjson.HjsonParser.error(HjsonParser.java:500)
at org.hjson.HjsonParser.expected(HjsonParser.java:494)
at org.hjson.HjsonParser.readStringInternal(HjsonParser.java:292)
at org.hjson.HjsonParser.readString(HjsonParser.java:282)
at org.hjson.HjsonParser.readValue(HjsonParser.java:117)
at org.hjson.HjsonParser.readObject(HjsonParser.java:199)
at org.hjson.HjsonParser.readValue(HjsonParser.java:119)
at org.hjson.HjsonParser.parse(HjsonParser.java:88)
at org.hjson.JsonValue.readHjson(JsonValue.java:130)
at de.softquadrat.jdbc.json.Test.main(Test.java:13)
The 0B character (Ctrl-K) in the generated Json is invalid. It should be quoted as \u000b. HJson is unable to parse the Json generated by itself and throws the ParseException.
We are using HJson to parse regular Json files and found an issue with control characters in Strings.
The input file looks like
{ "aaa" : "xxx\u000byyy" }and we parse it with HJson. When the resulting JsonValue object is written using toString() or toString(Stringify.FORMATTED) the output contains invalid control characters. It is expected that these control characters are quoted using \u again according to Json standard.
This example program shows the problem:
The output of the test program gives:
0000 0A 7B 0A 20 20 22 61 61 61 22 3A 20 22 78 78 78 .{. "aaa": "xxx 0010 0B 79 79 79 22 0A 7D -- -- -- -- -- -- -- -- -- .yyy".} Exception in thread "main" org.hjson.ParseException: Expected valid string character at 3:13 at org.hjson.HjsonParser.error(HjsonParser.java:500) at org.hjson.HjsonParser.expected(HjsonParser.java:494) at org.hjson.HjsonParser.readStringInternal(HjsonParser.java:292) at org.hjson.HjsonParser.readString(HjsonParser.java:282) at org.hjson.HjsonParser.readValue(HjsonParser.java:117) at org.hjson.HjsonParser.readObject(HjsonParser.java:199) at org.hjson.HjsonParser.readValue(HjsonParser.java:119) at org.hjson.HjsonParser.parse(HjsonParser.java:88) at org.hjson.JsonValue.readHjson(JsonValue.java:130) at de.softquadrat.jdbc.json.Test.main(Test.java:13)The 0B character (Ctrl-K) in the generated Json is invalid. It should be quoted as \u000b. HJson is unable to parse the Json generated by itself and throws the ParseException.