Search Results (4 posts found):
I've just started a little C programming project in order to improve my C programming and, indeed, to see if I'm actually clever enough to write something in C. The project I've chosen is a little ID3 tag editor for MP3 files.
I've chosen this because it might be useful and because it involves something I quite enjoy, which is parsing files. It's also the sort of project that doesn't need a GUI but can later be extended to have one.
I chose C because, as stated earlier, I wanted to prove to myself that I can do it and because it's a great language for byte level manipulation of the sort required to parse a binary file like an MP3.
MP3 files support two types of ID3 tag, these being version 1 and version 2 (of which there are 4 types). ID3v1 tags reside at the end of the mp3 file and are limited in length to 128 total bytes: 3 for the keyword "TAG", 30 each for the title, artist and album, 4 for the year, and then either 28 or 30 for a comment, and, if using a 28 byte comment, 2 for the track number, and finally 1 byte for the genre.
ID3v1 tags are therefore trivial to manipulate.
ID3v2 moved the tags to the start of the file and introduced the concept of separate frames of data, each to store one tag and its value.
The ID3v2 header starts with the values "ID3" (0x49 0x44 0x43 in hex) followed by the minor version number (0x03 for ID3v2.3, 0x04 for ID3v2.4, etc) and then the build number (usually 0), a byte for flags related to the ID3 tagss, and finally four bytes for the overall ID3 header size. Thus an ID3v2 header can be identified by looking like this:
49 44 33 vv vv ff xx xx xx xx
Where vv are the version bytes, ff is the flags, and xx xx xx xx is the big-endian header size.
When I first started writing the decoding function for the ID3 header I was reading the number of bytes as dictated by the xx xx xx xx in the header, but found I was reading well beyond the end of the header itself, causing all sorts of problems. This led me to a google search where I discovered the concept of "Sync-Safe Integers".
These sync-safe integers are not a universal computing concept, but rather one created purely for MP3 encoding, it seems. And this is where sychronisation safe integers come in.
In order for those four bytes indicating header size to not be confused for later bytes that are used to synchronise data frames they cannot contain the values "FF 00" and the way to do this is to ensure that no bytes not intended for syncronisation contain the value FF unless always guaranteed to be followed by a non-zero value, such as the byte order marker FF FE.
To do this, we insert a zero at bit 8 (the left-most bit, and most-significant bit) of each byte, working from least to most significant, and shift the rest of the bytes accordingly:
Thus the value:
00000001 00000110 00010100 11000001
becomes
00001000 00011000 00101000 01000001
This has the side-effect of reducing the capacity of our integer from being 32 bits to being 28 bits, but in the context of ID3 tags that limitation is irrelevant.
For my own program, I achieved this shift by using a bit mask to take each group of bits I want to operate on in turn, applying the mask as a bitwise AND to the integer, and then shifting the requisite number of positions to the right.
I then bitwise ORed the results together:
return ((in & 0x0000007F) |bit mask %00000000 00000000 00000000 01111111
((in & 0x00003F80) << 1) |bit mask %00000000 00000000 00111111 10000000
((in & 0x001FC000) << 2) |bit mask %00000000 00011111 11000000 00000000
((in & 0x0FE00000) << 3));bit mask %00001111 11100000 00000000 00000000
g a sync-safe integer from a regular integer.
The decoding sequence is just the reverse:
return ((in & 0x0000007F) | // bit mask %00000000 00000000 00000000 01111111
((in & 0x00007F00) >> 1) |bit mask %00000000 00000000 01111111 00000000
((in & 0x007F0000) >> 2) |bit mask %00000000 01111111 00000000 00000000
((in & 0x7F000000) >> 3)); // bit mask %01111111 00000000 00000000 00000000
In both cases, the ORed instructions are independent of each other in the sense that each has its bitmask applied and is shifted BEFORE all four results are ORed together.
And using this approach I was able to now correctly identify the length of the ID3 tags. Note, this value includes any padding at the end of the tags and before the audio data.
So that's sync safe integers - something you'll never encounter outside the realms of writing a decoder for ID3 tags.
One of the things that's come up several times in my Delphi experience is the need to Base 64 encode and decode data. Sadly, there's no built-in function to do this, but as Base-64 is a very straightforward encoding, it's a pretty trivial task to write your own.
Base-64 is a means of encoding binary data using the 62 characters comprising the lowercase alphabet, the uppercase alphabet and the numbers 0-9, as well as the characters + and / for a total of 64 different characters, plus = used for padding.
Why 64? Because the number 63 is the highest number that can be represented in 6 bits. 63 allows for 64 positions in a zero-indexed array, and as we'll see in a minute, the use of 6 bits is crucial to how Base-64 works.
The basic principle is this: take three 8-bit values and turn them into four 6-bit values. These values are used to index an array of the character set mentioned above, and the characters then form the output string.
For example, take the string "Cat". Assuming an ASCII encoding, this comprises the values 67, 97, and 116, the values for uppercase C, lowercase A and lowercase T, respectively. These are represented in binary thus:
01000011 01100001 01110100
Divided into groups of 6:
010000 110110 000101 110100
These become the numbers
16, 54, 5, and 52
And if we use these to index into the following array (zero-indexed):
ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/
We get the characters:
Q2F0
Decoding is just finding the index of the characters, converting those values to binary, grouping the bits in 8s and converting back to numerical values.
Earlier we said we were converting groups of three 8-bit bytes into groups of four 6-bit bytes, but what if the input data isn't divisible by four? Simple, we pad any partial groups with zeros and use the padding character = for any output characters that are missing entirely.
So, if our input was "Ca" instead of "Cat" we'd have input bytes of:
01000011 01100001
Grouped into sixes, we finish off the last group with zeros:
010000 110110 000100
Convert to the character sequence as before, and append the padding character:
Q2E=
Simple, right?
So why do we need the padding character? Even without it, we'd be able to unambiguously decode back to our original data. But what if we concatenated several base-64 values one after the other? If we couldn't guarantee that each one was a multiple of four we could find ourselves in the situation where the last bits of one string are concatenated to the first bits of another string and produce a string that is different to the input from that point on. The padding characters remove that possibility.
So how do we do this in Delphi?
Well, first it's important to remember that Base-64 can be used to encode **any** binary data, not just character strings, so it makes sense that the input for our encoding function is an array of bytes. If we do happend to want to encode a string, we can get the appropriately encoded byte values back from the string using the TEncoding class.
Then we loop through the string taking three characters at a time, keeping track of how many bits are left to read, and converting them to 6-bit bytes.
So far:
function Base64EncodeBytes(inputstring : TBytes) : string;
const base64set = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/';
// Our array index into the input string
// and the number of characters remaining
var i, rem: integer;
// The three characters of the input
// we're currently processing
i1, i2, i3 : byte;
// The four array indexes
// for our output characters
n1, n2, n3, n4 : byte;
begin
rem := Length(inputstring);
i := 0;
while rem > 0 do
begin
i1 := 0;
i2 := 0;
i3 := 0;
if rem > 0 then
begin
i1 := inputstring[i];
inc(i);
end;
if rem > 1 then
begin
i2 := inputstring[i];
inc(i);
end;
if rem > 2 then
begin
i3 := inputstring[i];
inc(i);
end;
So that's the 8-bit bytes gathered. Now we need to split these into groups of 6. The way I decided to do this was to make one long number by left shifting the byte values and then right shifting each group of 6 into the lowest 6-bit position to get the new byte values.
// Form a long number from which
// to take our groups of six
outputNo := (i1 shl 16) + (i2 shl 8) + i3;
// Shift the six left most bits
// over to the right by 18 places,
// so the occupy the six least-significant bits
n1 := outputNo shr 18;
n2 := (outputNo shr 12) and 63;
n3 := (outputNo shr 6) and 63;
n4 := (outputNo) and 63;
The fact our variables n1 - n4 are bytes means that the higher order bits we don't need are automatically truncated. All we do then is ensure that bits 7 and 8 are also both 0 by anding with 63 (binary 00111111) and we have our array indexes.
All that remains is to get the correct output characters, add any padding based on the number of remaining input characters and close off our loop.
if rem < 2 then
result := result + Base64Set[n1 + 1]
+ Base64Set[n2 + 1] + '=='
else if rem < 3 then
result := result + Base64Set[n1 + 1]
+ Base64Set[n2 + 1]
+ Base64Set[n3 + 1] + '='
else
result := result + Base64Set[n1 + 1]
+ Base64Set[n2 + 1]
+ Base64Set[n3 + 1]
+ Base64Set[n4 + 1];
rem := length(inputstring) - i;
end;
end;
Et voilá!
I've recently had the misfortune of having to integrate web-based payment software with Visa's Cybersource REST API using their Microforms integration. Unfortunately, the documentation is somewhat lacking and support is virtually non-existent.
Fortunately I'm a tenacious thing and with much trial and error managed to get it working. I'm posting this information here in the hope it saves someone else the hair-tearing it cost me.
First, let me explain what Microforms are: they are iframes hosted at Cybersource themselves but embedded on your site to collect the sensitive credit card data without you having to worry about PCI compliance. You request a "Server Side Capture Context" from the API and use this to initialise a javascript object which creates the fields on your page. When the data has been entered, a token is requested that contains the encrypted card data and then when all user processing has been completed, this can be sent on to Cybersource to confirm the payment.
The first step was to get an account on their Sandbox at
https://developer.cybersource.com/hello-world/sandbox.html
Even this step was painful, as my first five attempts at registration failed to send me a confirmation email with my account credentials, so I was unable to log in. And yes, I did check spam. Eventually, though, I did get an email and was able to log in and create a Shared Key.
The next problem I had was that the instructions for how to create the authentication header for messages to the API were vague, contradictory and incomplete. The documentation says that the message data need only contains a targetOrigins property, thus:
{ "targetOrigins": &["http://localhost"&] }
Only this isn't the case. If you only send this you get an error back from the API. It turns out you also need to request the fields you want to display in the microform too:
{
"fields\": {
"paymentInformation": {
"card": {
"number": { },
"securityCode": {
"required": false
}
}
}
},
"targetOrigins": &["https://localhost"&]
}
So that's the payload sorted out, now for the hard part: the Authentication header.
Cybersource gives you the choice of a JWT certificate or an HTTP signature. I'm using the latter because it seemed easier to set up. The signature is a hashed copy of all the headers being included in the message and a few additional bits. Deciding exactly what to include took me ages as when it didn't work there was no indication of what part I'd got wrong: the header, the signature, the payload, or something else? For brevity, I'm going to just post the PHP code that got this working (with my login credentials replaced, of course):
<?php
$url = 'https://apitest.cybersource.com/microform/v2/sessions';
$merchantId = 'testres';
$keyId = '08c94330-f618-42a3-b09d-e1e43be5efda';
$sharedKey = 'yBJxy6LjM2TmcPGu+GaJrHtkke25fPpUX+UY6/L/1tE=';
$host = 'apitest.cybersource.com';
$data =
"{
\"fields\": {
\"paymentInformation\": {
\"card\": {
\"number\": { },
\"securityCode\": {
\"required\": false
}
}
}
},
\"targetOrigins\": &[\"https://localhost\"&]
}
";
$hash = hash("sha256", $data, true);
$digest = base64_encode($hash);
$date = date("D, d M Y G:i:s")." GMT";
$sigstring = "host: apitest.cybersource.com\ndate: $date\n(request-target): post /microform/v2/sessions\ndigest: SHA-256=$digest\nv-c-merchant-id: $merchantId";
$sigstringbytes = utf8_encode($sigstring);
echo "$sigstring\n";
$rawkey = base64_decode($sharedKey);
$hashval = hash_hmac("sha256", $sigstringbytes, $rawkey, true);
$sighash = base64_encode($hashval);
$headers = array(
"Accept-Language: en-GB,en;q=0.5",
"Accept-Encoding: gzip, deflate, br",
"digest: SHA-256=".$digest,
"v-c-merchant-id: ".$merchantId,
"v-c-date: ".$date,
"date: ".$date,
"signature: keyid=\"".$keyId."\", algorithm=\"HmacSHA256\", headers=\"host date (request-target) digest v-c-merchant-id\", signature=\"".$sighash."\"",
"host: apitest.cybersource.com",
"Content-Type: application/json"
);
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_POST, true);
curl_setopt($curl, CURLOPT_HTTPHEADER, $headers);
curl_setopt($curl, CURLOPT_POSTFIELDS, $data);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
//curl_setopt($curl, CURLOPT_PROXY, "127.0.0.1:8888"); // Uncomment to see traffic in a traffic analyser like Fiddler
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
$resp = curl_exec($curl);
curl_close($curl);
echo $resp."\n";
?>
The tricky part was working out which headers to include and in what order. The headers here are all necessary (in my experience) to get anything other than a 401 error back from the API.
The digest is easy: it's just a base64 encoded version of a SHA256 hash of the packet data (the JSON request).
The order of the headers in the signature is also crucial, it would seem. Once all of this is set up correctly, a call to the endpoint should return a capture context in the form of a public key:
header.payload.signature
The whole thing should be passed to the javascript side of things to initialise the Flex() object and create the microform. From there, the card number and CVV (and expiry month and year) can be collected and then either the normal, non-3D Secure, endpoint can be contacted or the Payer Authentication endpoint can be used to perform proper security checks on the card. I'll cover that in the next post.
I recently needed to implement RSA cryptography in Delphi, sticking to a few requirements of my own choosing:
1. The library used should be free
2. The library license should be permissive enough that I can use the library in a commercial application
3. The library should have as few extra files to include as possible
I looked initially at LockBox, but it came with a multitude of files to either copy to my source folder or include in my library path. I don't want other developers in my team to have to add yet more third party libraries, so I gave this a miss. Additionally, the documentation wasn't great and finding example online was quite tricky.
I then tried PGPCrypt, but could only find a version of that for Lazarus, the free Delphi compiler.
Next was the TMS Cryptography pack, but this isn't free, so again a non-starter.
Then I thought about OpenSSL. We already use OpenSSL as a DLL for a different, but related, purpose so the file is already present in our source and program folder. What we don't have are function headers as we're calling the DLL functions by importing them directly (we only use three at present), but RSA is more complicated and uses a particular structure that would need replicating in Delphi. I thought this might mean OpenSSL was also a non-starter, but after much searching online I came across the Delphi-OpenSSL project at https://github.com/UniGe/delphiopenssl/ which consists really of only three .pas files, only one of which is necessary for what I'm trying to implement: RSA encryption and decryption. For this purpose I need use only the LibEAY32.pas file coupled with the LibEAY32.dll file. We're using version 1.0.2 of this, and the Delphi-OpenSSL project is compatible with that. Great!
I needed to make a few minor changes, as despite the programmers' best efforts it seemed confused about the difference between Char and AnsiChar in Delphi 2009, but the changes were minimal and consisted, almost entirely, of me replacing references to PChar or PCharacter with PAnsiChar.
The next challenge was to see if I could use an existing key to encrypt a file.
I added my modulus, public exponent and private exponent (not needed for encryption) as Base64 encoded string constants and used:
_rsa = RSA_New();
to create a new RSA structure to populate with the key information. This actually returns a pointer to an RSA object, so it needs both dereferencing and casting when used. The properties we're interested in are the modulus, n; the public exponent, e; and the private exponent, d (when decrypting, later).
These properties take the form of BigNum objects as they can be sometimes thousands of bits long. The BigNums need to be created before being used:
RSA(_rsa^).n := BN_New();
RSA(_rsa^).e := BN_New();
The Base64 values were then decoded and altered, one byte at a time, to hex using IntToHex (which returns a string) and then BN_HexToBN was used to set the RSA record's properties accordingly:
BN_HexToBN(RSA(_rsa^).n, ansistring(public_modulus));
BN_HexToBN(RSA(_rsa^).e, ansistring(public_exponent));
The input string can then be encrypted by calling:
enc_in := SysUtils.TEncoding.ASCII.GetBytes(string(input_text));
to get the string character values as a byte array, followed by:
RSA_Public_Encrypt(length(enc_in),PAnsiChar(enc_in), PAnsiChar(@textout),_rsa,RSA_PKCS1_PADDING);
to do the actual encryption. The output cypher text is held in textout. This will look like garbage but can be converted to a Base64 string for convenient storage.
And that's it.
Two caveats:
I had to add an extra property to the RSA record to match the version of OpenSSL we're using. This was:
engine: pointer;
below the meth property.
And I had to initialise the SSL library by calling:
SSLEay_version(_SSLEAY_VERSION);
before making any calls to encryption or decryption routines. There's probably a better way to do this, but this did the trick.
I put this information here in the hope that it helps someone else who struggled as much as I did to implement this!