Skip to main content

Unicode -- Uphill both ways (Ruby Programming pt. 7)

I found this cool article on Unicode (it gets UTF-16 wrong but that's ok). However I'm running into a large wall dealing with Unicode in my program. So I'll put it out there so a solution presents itself.

So far my program checks each line of the file to see if it's ASCII only text. If so it reverses it with Ruby's built-in reverse method.

If not what I want to do is to have it read each hex pair (or four-some) decide if it is below U+007F (inclusive) to treat it as plain ASCII and pass the character as one element to an array, if it's between U+0080 and U+FFFF then to take a two byte chunk and pass it as one element to an array. And finally if it is between U+010000 and U+10FFFF then to take a three byte chunk and pass it as one element to an array. Then to read the elements of the array First one In Last one Out (FILO), remove the end of line (/n) marker and put the elements into another array. Join that array add an end of line element and write it to the file.

So the first thing I need to do is find a way of reading the hexadecimal values of the characters. So after a lot of looking I found a hex editor plugin for Notepad++ and though it doesn't do exactly what I want I figure something out. The last character or the U+007F is 7F in the hex value of the file. Apparently Notepad++ hides the 00 of endian-ness. So that's the one I want to move as a one element to another array. And at least for now I can assume that every thing above 80 is a two-byte element, till I figure a way of reading the three-byte ones. It won't be perfect but if it works it will be a step.
Now to try it out.

Comments

Popular posts from this blog

Multiple Assignments in Ruby

In this Rails project I'm running I need to set a whole bunch of individual variables to 'disabled' at once depending on the group the current user belongs to. We can use multiple assignment in this case but with a caveat. Multiple assignments are a quick way of setting a lot of variables at once. h, k = 1,2  => [1,2] h => 1 k => 2 but   h, k = 1      => 1 h => 1 k => nil In that case we can do this: h = k = 1   => 1 h => 1 h => 1  Beware this sets them to the same object. So if you change one you change both. h = k = [] h << 1 h => [1] k => [1] While we're at it here is how to reverse two variables: h = 1 k = 2 h,k = k,h h => 2 k => 1

What Medieval Economics can teach us about tariffs.

As a teen, I used to play Dungeons and Dragons (D&D) with my friends. This started an interest in the medieval period that led to me taking a medieval history class in college just to understand the period more. Over the years I've also read great books like " Dungeon, Fire and Sword " about the crusades (I recommend the book) and yet with all that knowledge it wasn't until recently that it occurred to me I had a completely wrong understanding of economics in the Medieval Period. "Viking helmets, sword and footwear" by eltpics is licensed under CC BY-NC 2.0 In my D&D games, players who are adventures battling monsters and creatures would need equipment and on the trips to town, they'd get resupplied with their adventuring necessities. I'd run these moments referencing my imagination of what it must have been and fantasy books I'd read. There be an inn with a raucous bar, a gruffly black-smith, if a city also a weapon and armor sm...

Setting up the Marble Mouse for Ubuntu

I have a Logitech trackball which I love. I bought it when my wrists started hurting after hours of long Photoshop sessions. The switch to the trackball fixed all that. It allows me to hold my hand in the perfect position and it's great for most browsing with one exception: It lacks a middle-click wheel. This presents a bigger problem than it seems, as design seems to be removing the scroll bars at the edges of windows in favor of small slits. So to get around this you can emulate a scroll wheel or use the middle click to activate a scroll motion on various programs. Logitech has a utility that sets this up in Windows but in Ubuntu you have a better (if more hard-work required) option. The Ubuntu option is better because unlike in Windows you can emulate the wheel with the track-ball by holding down a button. This is the option I like to use. In Ubuntu you write a configuration file to set up the trackball to work the way you want it. I have to re-create this...