Skip to content

HJson fails to quote control character with \uXXXX in String #34

Description

@treimers

We are using HJson to parse regular Json files and found an issue with control characters in Strings.

The input file looks like

{ "aaa" : "xxx\u000byyy" }

and we parse it with HJson. When the resulting JsonValue object is written using toString() or toString(Stringify.FORMATTED) the output contains invalid control characters. It is expected that these control characters are quoted using \u again according to Json standard.

This example program shows the problem:

package de.softquadrat.jdbc.json;

import java.nio.charset.StandardCharsets;

import org.hjson.JsonValue;
import org.hjson.Stringify;

public class Test {
	public static void main(String[] args) {
		JsonValue hjson = JsonValue.readHjson("{ \"aaa\" : \"xxx\\u000byyy\" }");
		String s = hjson.toString(Stringify.FORMATTED);
		System.out.println(hexdump(s));
		JsonValue.readHjson(s);
	}

	// from https://gist.github.com/geekman/5243537
	public static String hexdump(String s) {
		byte[] data = s.getBytes(StandardCharsets.UTF_8);
		final int perRow = 16;
		final String hexChars = "0123456789ABCDEF";
		StringBuilder dump = new StringBuilder();
		StringBuilder chars = null;
		for (int i = 0; i < data.length; i++) {
			int offset = i % perRow;
			if (offset == 0) {
				chars = new StringBuilder();
				dump.append(String.format("%04x", i))
					.append("  ");
			}
			int b = data[i] & 0xFF;
			dump.append(hexChars.charAt(b >>> 4))
				.append(hexChars.charAt(b & 0xF))
				.append(' ');
			chars.append((char) ((b >= ' ' && b <= '~') ? b : '.'));
			if (i == data.length - 1 || offset == perRow - 1) {
				for (int j = perRow - offset - 1; j > 0; j--)
					dump.append("-- ");
				dump.append("  ")
					.append(chars)
					.append('\n');
			}
		}
		return dump.toString();
	}
}

The output of the test program gives:

0000  0A 7B 0A 20 20 22 61 61 61 22 3A 20 22 78 78 78   .{.  "aaa": "xxx
0010  0B 79 79 79 22 0A 7D -- -- -- -- -- -- -- -- --   .yyy".}

Exception in thread "main" org.hjson.ParseException: Expected valid string character at 3:13
        at org.hjson.HjsonParser.error(HjsonParser.java:500)
        at org.hjson.HjsonParser.expected(HjsonParser.java:494)
        at org.hjson.HjsonParser.readStringInternal(HjsonParser.java:292)
        at org.hjson.HjsonParser.readString(HjsonParser.java:282)
        at org.hjson.HjsonParser.readValue(HjsonParser.java:117)
        at org.hjson.HjsonParser.readObject(HjsonParser.java:199)
        at org.hjson.HjsonParser.readValue(HjsonParser.java:119)
        at org.hjson.HjsonParser.parse(HjsonParser.java:88)
        at org.hjson.JsonValue.readHjson(JsonValue.java:130)
        at de.softquadrat.jdbc.json.Test.main(Test.java:13)

The 0B character (Ctrl-K) in the generated Json is invalid. It should be quoted as \u000b. HJson is unable to parse the Json generated by itself and throws the ParseException.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions