I recently had notice from my VPS provider that my site was using a compromised version of PHP that contained the php2shell vulnerability, so reported on by the media lately.
Being the responsible website owner that I am, I immediately set about to upgrade my server from Ubuntu 20.04 (Focal Fossa) to the next LTS version 22.04 (Jammy Jellyfish). Arguably I could have gone further, but as Jammy is still under long term support, and because the upgrade process is rarely smooth, I opted to only go up one version. The necessary patch for PHP 8.1 is provided by Canonical for this version, so it'll do.

And that's where the fun started. I did all the usual prep for a version upgrade, including bringing existing packages up to the latest available version within the installed Ubuntu release, and then I set the upgrade command going. It got most of the way before falling over on MySql server - the database that underpins my blog posts.
This stopped the upgrade process in its tracks, and was caused by a lack of available RAM. Sadly, my server is still surviving on just 512MB of RAM. The solution suggested by Claude (yuck...) was to use a ramdisk to overcome this shortfall. This worked. Sort of. It overcame the RAM problem, but the aborted installation from earlier left MySQL in an unupgradable position, from which I was likely to lose data.

The position I found myself in was that I still had access to the innodb files that contain the blog data but no means to access them. At this point I decided that if MySQL was going to be a resource hog on my overstretched (virtual) machine, then I'd have to do without it.

The solution I settled on was to migrate everything over to SQLite. Still a relational database and a syntax I'm familiar with, but much less resource hungry and perfect for a tiny blog with infrequent updates, only one contributor, and hardly any visitors. The question was how to extract the data from the MySQL database without running up an instance of MySQL server. The answer, not suggested by Claude, is an excellent little Python program called ibd2sql which can extract both the DDL (the code that creates the tables) and the data as a series of INSERT statements and output them as a file in MySQL syntax.

Thankfully the site only comprises a small handful of tables, as the website and database are hand-rolled rather than relying on third party software like WordPress, so a quick bash script to loop through the ibd files and create SQL output did the job, and ChatGPT (again, yuck...) converted them from MySQL syntax to SQLite syntax. The differences aren't important. This allowed me to import all my existing blog posts back into my new SQLite database. Hooray! The next step was to modify the PHP to use the new database instead of the old. There were a few data retrieval functions (mostly fetch_object()) not available, but it wasn't hard to convert these to fetchArray() to get everything up and running again.

The result is that I can now remove MySQL from the server entirely, and have a smaller resource footprint going forward.

I'm also no longer vulnerable to the php2shell bug. Imagine the damage an attacker could wreak if they access this server! :D

ibd2sql is available from here and is an invaluable piece of software if you find yourself in the same situation as me.

I've recently had the misfortune of having to integrate web-based payment software with Visa's Cybersource REST API using their Microforms integration. Unfortunately, the documentation is somewhat lacking and support is virtually non-existent.

Fortunately I'm a tenacious thing and with much trial and error managed to get it working. I'm posting this information here in the hope it saves someone else the hair-tearing it cost me.

First, let me explain what Microforms are: they are iframes hosted at Cybersource themselves but embedded on your site to collect the sensitive credit card data without you having to worry about PCI compliance. You request a "Server Side Capture Context" from the API and use this to initialise a javascript object which creates the fields on your page. When the data has been entered, a token is requested that contains the encrypted card data and then when all user processing has been completed, this can be sent on to Cybersource to confirm the payment.

The first step was to get an account on their Sandbox at https://developer.cybersource.com/hello-world/sandbox.html
Even this step was painful, as my first five attempts at registration failed to send me a confirmation email with my account credentials, so I was unable to log in. And yes, I did check spam. Eventually, though, I did get an email and was able to log in and create a Shared Key.

The next problem I had was that the instructions for how to create the authentication header for messages to the API were vague, contradictory and incomplete. The documentation says that the message data need only contains a targetOrigins property, thus:

{ "targetOrigins": &["http://localhost"&] }

Only this isn't the case. If you only send this you get an error back from the API. It turns out you also need to request the fields you want to display in the microform too:

{
"fields\": {
"paymentInformation": {
"card": {
"number": { },
"securityCode": {
"required": false
}
}
}
},
"targetOrigins": &["https://localhost"&]
}

So that's the payload sorted out, now for the hard part: the Authentication header.

Cybersource gives you the choice of a JWT certificate or an HTTP signature. I'm using the latter because it seemed easier to set up. The signature is a hashed copy of all the headers being included in the message and a few additional bits. Deciding exactly what to include took me ages as when it didn't work there was no indication of what part I'd got wrong: the header, the signature, the payload, or something else? For brevity, I'm going to just post the PHP code that got this working (with my login credentials replaced, of course):

<?php

$url = 'https://apitest.cybersource.com/microform/v2/sessions';

$merchantId = 'testres';
$keyId = '08c94330-f618-42a3-b09d-e1e43be5efda';
$sharedKey = 'yBJxy6LjM2TmcPGu+GaJrHtkke25fPpUX+UY6/L/1tE=';

$host = 'apitest.cybersource.com';

$data = 
"{
\"fields\": {
\"paymentInformation\": {
\"card\": {
\"number\": { },
\"securityCode\": {
\"required\": false
}
}
}
},
\"targetOrigins\": &[\"https://localhost\"&]
}
";
$hash = hash("sha256", $data, true);

$digest = base64_encode($hash);

$date = date("D, d M Y G:i:s")." GMT";
$sigstring = "host: apitest.cybersource.com\ndate: $date\n(request-target): post /microform/v2/sessions\ndigest: SHA-256=$digest\nv-c-merchant-id: $merchantId";
$sigstringbytes = utf8_encode($sigstring);

echo "$sigstring\n";

$rawkey = base64_decode($sharedKey);
$hashval = hash_hmac("sha256", $sigstringbytes, $rawkey, true);

$sighash = base64_encode($hashval);


$headers = array(
"Accept-Language: en-GB,en;q=0.5",
"Accept-Encoding: gzip, deflate, br",
"digest: SHA-256=".$digest,
"v-c-merchant-id: ".$merchantId,
"v-c-date: ".$date,
"date: ".$date,
"signature: keyid=\"".$keyId."\", algorithm=\"HmacSHA256\", headers=\"host date (request-target) digest v-c-merchant-id\", signature=\"".$sighash."\"",
"host: apitest.cybersource.com",
"Content-Type: application/json"
);


$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_POST, true);
curl_setopt($curl, CURLOPT_HTTPHEADER, $headers);
curl_setopt($curl, CURLOPT_POSTFIELDS, $data);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
//curl_setopt($curl, CURLOPT_PROXY, "127.0.0.1:8888"); // Uncomment to see traffic in a traffic analyser like Fiddler
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);

$resp = curl_exec($curl);

curl_close($curl);

echo $resp."\n";

?>

The tricky part was working out which headers to include and in what order. The headers here are all necessary (in my experience) to get anything other than a 401 error back from the API.

The digest is easy: it's just a base64 encoded version of a SHA256 hash of the packet data (the JSON request).

The order of the headers in the signature is also crucial, it would seem. Once all of this is set up correctly, a call to the endpoint should return a capture context in the form of a public key:

header.payload.signature

The whole thing should be passed to the javascript side of things to initialise the Flex() object and create the microform. From there, the card number and CVV (and expiry month and year) can be collected and then either the normal, non-3D Secure, endpoint can be contacted or the Payer Authentication endpoint can be used to perform proper security checks on the card. I'll cover that in the next post.

I've realised that this blog has strayed somewhat from my original aim of it being about my journey of learning Welsh, and more just my opinion on various language-related things.

So, in a return to its true purpose, this post is a review of the book "Yn Ei Gwsg" (In His Sleep) by Bethan Gwanas.

Dafydd walks in his sleep. It's part of the reason, he suspects, for the breakup of his marriage and it gets him into all sorts of odd situations with his neighbours. One morning, however, it takes a more sinister turn when he wakes up covered in blood - not his own - and sets off with his faithful dog Wmffra to follow the bloody footprints to find out to whom the blood belongs.



The book is not nearly as dark as that synopsis makes it sound. It's actually very light-hearted, humorous and a touch sad in places. Bethan Gwanas writes novels for native Welsh speakers as well as learners, and I think that helps her keep this novel simple yet clearly aimed at adults. The language used has clearly been thought about so that the level remains suitable for the target audience, yet also shows a progression from start to finish and introduces new words and phrases.

I really enjoyed reading it. Unusually for me, I found I could hardly put it down. I think this was partly because it was genuinely an enjoyable story, and partly because the level was pretty much spot-on for my ability (this is set as Lefel Sylfaen) so I found I was able to read straight through most pages without having to resort to a dictionary (the glossary of new terms at the bottom of each page, as is common for Welsh-for-adults books, helps enormously). The language used is more northern in style, but I didn't find this a hindrance at all - in fact it helped to cement some knowledge of northern dialect that I'm bound to encounter in the wild anyway.

The last book I read before this was "Ffenestri" by Lois Arnold. If you've read up to the end of the Sylfaen level in that, you shouldn't have any problems with this and I'd recommend it as not only a good book for learners, but a good read in its own right.

Ok, so it's been about seven months since I started learning Welsh in earnest, and he's what I think is hard and what I think is fairly easy about Welsh:

1. Mutations. Letters change at the start of one word depending on what word they follow. However I'm finding that this starts to feel more natural in most circumstances after a while.

2. Versions of "about" and "of/from" for each pronoun. If you're talking about a thing, they're "am" and "o" respectively. But if you're talking about/from me, you, her, him, us or them you're looking at "amdana, amdanat, amdani, amdano, amdanon and amdanyn" and "ohono, ohonot, ohoni, ohono, ohonon and ohonyn"!

3. The letter "Ll". Yes, it counts as one letter. The pronunciation is somewhere between ell and hissing like an angry goose with a headache. For some reason I find it easier to pronounce at the start of a word, like Llawr, than in the middle or end of a word, like Arall or Galla.

4. Conjugations. Well, this is the same for any language, really. Remembering the conjugations of short-form verbs is just one of those things you have to get used to.

5. Yes and no. There are no words for Yes and No in Welsh, you just repeat back (or negate) the appropriate conjugation of the verb in the question. "Do you like cheese?", "I do". "Are you tired?" "I am". "Are you reading?" "I am not". But that's only in the present tense. In the past tense it's usually "Do" for yes and "Naddo" for no. But then for emphatic questions it's "Ie" for yes and "Nage" for no. If the question was asking if there is something with you (see below) you reply with Oes or Nag Oes. Remembering which to use is the hard part.

6. There aren't many words that are similar to other European languages. There are some, mostly more modern words borrowed from English: tacsi, parcio, ffatri, ffôn, beic, etc; and some ecclesiastical words like eglwys (church) and capel (chapel) but not to the extent that you find with Romance or Germanic languages.

Things that are easier than I expected:

1. Mutations. Yes, they're difficult, but actually you get used to them, and in many cases the rules are quite consistent. You just get used to dropping the G in words like gweithio and gweld when they follow a sentence subject. You get used to changing D to Dd in adjectives after "yn", and you get very used to nasal mutations after "fy".

2. Conjugations. Although remembering them is difficult, many of the conjugations are quite consistent and actually fit in quite well with the pronoun they go with.

3. Possession. I've said before about there being no form of "I have a (something)" in Welsh, they instead say something is "with them": Mae ci gyda fi - "There is a dog with me". I got the hang of this much quicker than I thought. I think you just change your way of thinking and everything slots together.

4. The language as a whole. I'm only a beginner. Very much so. But I've been surprised at how much I've covered in such a short space of time. I'm not proficient by any stretch of the imagination, and I can only string together fairly simple sentences, but on the other hand I can make many more types of sentences in Welsh than my partner's daughter can in Spanish; a language she's been learning at school for two years now. I can use the present tense, the past perfect, a sort of imperfect, the future in several forms and some conditional tenses and even a kind of limited subjunctive mood.

On the whole, Welsh has (so far) been easier than I expected. It's not the incomprehensible jumble of letters it always looked like on road signs when we used to visit my Grandma in Rhossili when I was a child. In fact, in many ways it's a lot easier than the languages I'd been taught at school.

My last post was about starting the Say Something In Welsh course. This is just an update on how that's going.

First, the main difference between this and DuoLingo is the sense of community. The DuoLingo forums (if you can call them that) are really only for people asking questions about the lessons themselves, or querying the correct answers to questions within the lessons. The Say Something In Welsh forums have much more spirit about them. People joke with each other, people ask questions, get replies and then go off topic about their experiences of learning Welsh. People share their real world experiences of *using* Welsh and receive encouragement from others for doing so. And, most importantly, the course creators and contributors also join in with all of this. And there's never a hint of condescension, just encouragement. This tone is maintained through the lessons themselves, with one of the later sentences on the very first lesson being "I'm not trying to speak Welsh, I am speaking Welsh" (Dw i ddim yn trio siarad Cymraeg, dw i'n siarad Cymraeg!)

Second, the course takes some getting used to for someone like me who has done a little Welsh before. The pronunciations are a little more colloquial, and words are omitted in the same way that a native speaker might: "Wyt ti'n bwyta" (you are eating) becomes simply "ti'n bwyta", in much the same way as an English speaker might ask simply "You eating?"

Then there's the manner of the course. Everything is very relaxed. You're told "don't worry about this now" when introduced to things like mutations and the long form of the past tense. You're told, "To form the past tense, you only need to know how to say "ness ee". This stumped me, until I realised it was just a colloquial pronunciation of what I know as "Mi wnes i". I'm not sure how well I cope with this casual approach. I **need** to know conjugations, I **need** to know why things are what they are. It's very hard for me to just let go and accept that things are as they are, and it will all become clear later.

But I'm trying. The course clearly works, as plenty of people on the forums are using it, and Iestyn is an excellent teacher. His lessons are well measured, his voice is clear, he's encouraging throughout, and he participates in the forums to answer questions and give advice.

He is clearly passionate about Welsh, and that enthusiasm is infectious.

Sometimes I see people on the DuoLingo forums say that they ultimately want to become fluent in the language they're studying. That strikes me as a lofty ambition.

For someone living in the country of the language they're learning, fluency as a goal makes sense, and in my opinion is probably achievable. Most language learners and teachers agree that the best way to become fluent is to immerse oneself in the language, and the best way to do that is by living in a country where it's spoken.

For someone not living in a country where their language is spoken, fluency must surely be much harder to achieve. Independent study, including reading and watching television programmes in the target language, can increase proficiency (I can't imagine there's such a thing as bad exposure to a language) but ultimately the way to fluency is to hear native speakers. Language, after all, is defined by its use as much as its rules. Language learning with no exposure to its actual use can, however, lead to the sort of hilarity found in the classic English As She Is Spoke.

I think goals need to be achievable and realistic in order to avoid discouragement. It's better to set a low bar, reach it, and then move on to the next target with a sense of acheivement than to reach too high and become discouraged at an apparent lack of success.

My goal with Welsh is definitely not to become fluent. Not at the moment, anyway. I don't think I'd stand a chance without being surrounded by Welsh speakers and using it every day. My aim is simply to be good enough to understand all the Welsh signs when I'm visiting Wales, to be able to conduct a simple transaction in a shop (actually I'm not that far off, if all I want is a coffee!) and to be able to read novels aimed at learners.

If you're learning a language, one of the best ways to gain exposure to that language and increase fluency is to visit a country where that language is spoken.

Herein lies the problem with Welsh: not all Welsh people speak it. They all speak English, but only around 20% of the population speaks Welsh. This means that if I'm in Wales, I can't initiate a conversation in Welsh because I can't guarantee that the other person will be a Welsh speaker.

Such a situation, where a native does not speak the native language, would be inconceivable in most other countries.

The fact that there are no monoglot Welsh speakers remaining also doesn't help. Because everyone speaks English fluently there's less incentive to make the effort to learn and speak Welsh.

Then there's the problem with signage. In France, other than at some touristy areas, all signs are in French. Signs alerting me to the "dèrniere sortie avant péage" will do so only in French. Translating and understanding the signs becomes part of the learning experience. Unfortunately, in Wales most signs are bilingual. The order of the languages is left to the discretion of the local authority, but an English translation will be on the sign somewhere. On the one hand, this allows you to quickly confirm that your translation of the Welsh was correct. On the other, it takes a lot of the fun and the necessity out of translating in the first place. It's very hard not to notice the English accidentally before you've had chance to attempt the Welsh.

What I have discovered, to my surprise and delight, is that several of my colleagues speak varying levels of Welsh. Remember, this is Somerset, England, not the north of Wales, so it's not entirely expected. One even speaks fluent Welsh, having attended a Welsh medium school. And he's said he's always happy to help with my Welsh learning.

So things are looking up.