-
Notifications
You must be signed in to change notification settings - Fork 105
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ASCIIHexToBytes’ Encoded Length Is Inconsistent With Its Decoded Length #326
Comments
Hey! Thanks for such a detailed research.
It works well when we have HEX tags (or BerTLVTag) like: Spec = &field.Spec{
Length: 1535,
Description: "Customer Related Data",
Pref: prefix.Binary.LL,
Tag: &field.TagSpec{
Length: 1,
Enc: encoding.ASCIIHexToBytes,
Sort: sort.StringsByHex,
},
Subfields: map[string]field.Field{
"01": field.NewComposite(&field.Spec{ // DataSet ID
// ... but because String, Numeric and Binary fields use value length, There is no reasonable way to solve that with the code for mentioned fields and We have
P.S. I tried to fix some places to avoid issues like this here #329 and here #328. |
* make it easier to see EMV tags * use field.Hex to avoid ASCIIHexToBytes packing issue (#326)
Hi, thank you very much for the great details! Those answer all my questions. I'll use something like a custom encoder to handle my use case instead of directly using the |
Overview
Version:
v0.22.1
It seems like the
ASCIIHexToBytes
encoder's packer produces a different data length value than what the unpacker needs, which makes the unpacking process fail with the "not enough data to read" error. I have a use case to store a field as a variable hex string so that it is easy to convert between Hex strings and binary data based on the variable length part.Example
Let's define a variable length field as the following:
Now, let's use the field to unpack binary data with a length of 3 bytes and its binary payload of 0x012345. Therefore, the raw binary input is defined as follows:
Now, use the
myField
defined above to unpack the input data and read the parsed string out of it.Let's now use the
myField
to pack the data that has just been unpacked to see if it produces the same original data.This instead produces a binary sequence of
0x0006012345
instead of the original0x0003012345
. The 2-byte variable length part becomes twice the original.Potential Causes
The
Unpack
function of thehexToASCIIEncoder
is currently defined as follows:iso8583/encoding/hex.go
Lines 78 to 95 in 55330b0
According to the way the
length
parameter is used in the function by using its value to represent how many bytes to read from the input and creating an output array with size twice its value, this means that thelength
here represents input size in bytes not how many Hex characters required to represent the data. This is different from, e.g., BCD, whoselength
represents how many BCD digits (half-byte) in the data part not how many bytes.Therefore,
0x0003012345
can be parsed correctly, as0x0003
length indicates 3 bytes in the following payload part.However, when performing packing using the same
myField
instance, the produced length part becomes0x0006
instead. According to the default packer logic shown below, the length part is produced by thespec.Pref.EncodeLength(spec.Length, len(value))
, which uses the length of the data before encoding. This means that the length part will always be twice the actual number of bytes because thelen(value)
represents the number of Hex characters, not size in bytes.This would break the message format if this configuration is used.
iso8583/field/packer_unpacker.go
Lines 8 to 21 in 55330b0
Questions
ASCIIHexToBytes
is also being used in multiple places, such as inBerTLVTag
, and seems to work fine. It could be tricky to change the behavior since it would impact other places as well.ASCIIHexToBytes
format represent the actual size in bytes of the data part or number of Hex characters in the data part, just like how BCD behaves?The text was updated successfully, but these errors were encountered: