?

Log in

No account? Create an account
In which Wade rambles about absurdly geeky things not even a… - Polyamorous Secular Transhumanist Me [entries|archive|friends|userinfo]
Wade

[ website | wade ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Links
[Links:| Polyamory Transhumanism ]

[May. 11th, 2007|02:29 pm]
Wade
In which Wade rambles about absurdly geeky things not even a sapio-sexual could love.

[Warning: Involves a collection of images of a tastefully nude girl and another collection collection of images of a centaur and not-so-tastefully-nude-but-darn-sexy girl. May or may not be work safe, depending on what kind of work you do :-)]

This is an example of one of those intellectual questions that I tend to get ... distracted by :-) Regardless of whether I have more important things that need to be done, and especially if I'm looking for a way to procrastinate.

What makes this kind of question all the more compelling is that, without actually doing some work, one can never really know how good the results will be. Curiousity killed the wade.

I don't remember why this thought occurred to me, and I make no claims that it is remotely original (I'm quite certain that other people have done exactly the same thing). Nonetheless, I hadn't thought of it before, so it was new to me. One day last week, while playing with some image manipulation software, I wondering how good of an approximation one could make of a image (photo, painting, etc.) using only text characters. This thought led to some simple feasiiblity analysis (which is always fun stuff). The basic strategy for doing this kind of thing is quite simple, and it felt like something that I could get some results from in a few hours of work (seeing results is always a good thing too) All of which made it even more difficult to resist the temptation to try it, instead of cleaning my apartment or ... applying for jobs :-)

The Problem

Here's the goal. We want to write a program that will accept an arbitrary image as input, and output a sequences of text characters (organized into rows and columns) that "most closely" approximates that image. There isn't anything particularily useful about this program, but the utility of something isn't really all that important; it is the enjoyment in solving the problem that is most of the fun :-) That said, I was really curious how good of an approximation could be made.

The Solution

Here are the steps involved in converting an image to text.

  1. Start with some image. For example, this one.


    g-grey.jpg
  2. Keep in mind that we are trying to approximate the image with a sequence of text characters, and text characters consist only of a foreground color (i.e. black) and a background color (i.e. white). Because of this, the first thing we need to do is convert the image to a black & white version that is as close to the original image as possible. There are all sorts of technical details associated with this (converting color to greyscale, converting grey-scale to black & white, use of dithering to make the black & white image look as close to the greyscale and original images as possible, etc.), but an image manipulation software package like Image Magick makes this sort of thing trivial for anyone to do.


    g-bw.jpg
  3. So, now that we have an image consisting solely of black and white pixels, we divide the image into tiny little rectangles the same size as a text character. But that poses an immediate problem, because "size of a text character" is entirely dependent on the font chosen (and, usually, on the individual character in question). Thankfully, there are certain fonts, called fixed-width fonts, in which every character as exactly the same width as every other character. For example, the font -misc-fixed-medium-r-semicondensed--13-120-75-75-c-60-iso8859-1 (forgive the awkward name; it is the standard way to fully specify a font in unix land) has characters that are all exactly 6 pixels wide and 13 pixels high. In fact, I am currently writing this LJ entry is an editor that is using exactly this font to display the text (fixed-width fonts are much more useful in programming contexts than variable width fonts).

    As an aside, here is a comparision of a variable-width font (the one that LJ uses for displaying text in your profile):

    abcdefghijklmnopqrstuvwxyz
    ABCDEFGHIJKLMNOPQRSTUVWXYZ

    Here is an example of text in a fixed-width font.

    abcdefghijklmnopqrstuvwxyz
    ABCDEFGHIJKLMNOPQRSTUVWXYZ

    Note that in the second version, each character takes up the same amount of space, whereas in the first one, different characters take up totally different amounts of space. The latter font allows us to make a "grid" of characters, whereas the former font does not.

    So now, back to our project. It is obvious why using a fixed-width font makes our lives easy; by dividing our image into little 6x13 squares, we can compare each such square against each character in the fixed-width font looking for the character whose sequence of black and white pixels most closely matches that of the square in question.

    The following image shows our black & white image divided into 6x13 rectangles (the grid pattern is not seen by the program when it is analyzing images - the grid is only for illustration purposes).


    g-grid.jpg
  4. Now we arrive at the crucial step in our handy dandy little program. We have a 6x13 rectangle, consisting of 6 * 13 = 78 pixels. Each pixel is either black or white. Furthermore, every character in our fixed width font is also 6x13, consisting of 6 * 13 = 78 pixels, each of which is either black or white. If we happened to know the sequence of black and white pixels for each character in the font, and could establish some way of comparing how "similar" each such character was to an arbitrary rectangle within our input image, we would be able to approximate that 6x13 rectangle within the image using the "best character" (the one whose sequence of black and white pixels most closely matches the rectangle itself).

    With a little bit of hackery, it is straight-forward to obtain the sequence of black/white pixels for each character in the font, and the sequence of black/white pixels in a particular 6x13 rectangle within the image. A little more hackery allows us to establish the best match. If we do this for every rectangle, and arrange the resulting characters into rows and columns corresponding to the rectangles in our image, we obtain our "best approximation" of the image using text alone.

    Sadly, some technical problems arise, because LJ doesn't allow me to embed CSS information into my entries, so I cannot force the following sequence of characters into being displayed in the right font. This means that what you see below isn't "as good" as it should be (the program finds the best matches for rectangles by assuming a specific font, and makes no guarantees about how it will look in another font. With that caveat in mind, here is our result.

                               _.-           
                                             
                                             
                    __                       
                 .#EN~~3L                    
                 #@@y)=#@b                   
                 B@@C ..4@@.                 
                dgN@@@yd@@$                  
                BB@$*  SB@#                  
                @@#'   &DBX&  ;              
             %.  `   3.\$<NP   ,             
              `    dy@"x7A#FY,   -`'t?       
          !v=     _@F L. `=L=,[g'' _?        
                  (<RNNNy_ " r^ `X           
                 :$P*^   x      =#=~         
                     ,_.' +.                 
               .a.ggDB5bx.  *.               
           W  _dN@@@@@@N@4J4  ", +           
      _.   `"b####N#F^ ``2Nbg@g.\_           
      7####mmmyxgv_J_`jQjyy@@P*#@@Ep`-       
                  ``~"~**P#N##$#Wg?#xg       
                                   "**i_   =j
                                          "'~
    

    g.txt

    In order to show you what the "text" looks like in the font I used, I can be sneaky, and actually convert the text back into an image. This also makes it possible to do the morphing discussed below.

    At one level, I'm happy with the results, especially given how trivial it was to create the program. On another level, I'm dissatisfied, and am quite certain I can do much better (see below for some discussion on this).


    g-txt.jpg

    Morphing is a standard image manipulation "trick" that allows one to smoothly convert one image into an arbitrary other image. When "animated", this can produce some neat effects.


    g-morph.gif

Example 2

Here's another example:

The original image


cm.jpg

The original file, before modifications
Grey scale image


cm-grey.jpg

Converted to greyscale
Dithered Black & White


cm-bw.jpg

Converted to black & white in such a way that the result looks as close as possible to the grey-scale image (a special image processing technique known as dithering is used to do this)
Grid shown


cm-grid.jpg

The image divided into tiny little 6x13 rectangles so that each such rectangle can be compared against each character in the chosen font. Note that the program does not see the grid (it is only for illustration purposes)
Text Version

@@@@@@[BNBBBBB3>*'                           ,#;g
@@@@@N@5gB$5&^'     yF4B. _)ggk            .ysxzj5
@@@#BNB[^('^       .'-m=5_@2Q.B[          TXM]3#
MBBW#@F7|   ___   .@bnLa@M@L $d@          ,~#2!&<1
@BXBNW'       ^#0@B* ~AT*"``=$@1         +v"2kyBy<
#MMZ]D.      ' ~LAL_pC.yg#k    4        x-'x<0?2$y
3Q%9x^   n      _@@_.L.jD# _.gE[         '! _SK%&2
}}<&8w9*(      aF [#+"*D#U$#?^EF__,g_      .j*>%#7
#{>2^q#<"    ''1  BA\z@M#J@[ _{a4 @@@@p y._ B@N9CY
$#%@, )#      ]  .[%BNBBN @@k'?  4N@@B__.#ggRg#GLp
@N6gB#2%)    ._,.y#[@5EB%pyB+@BF*]#*P~"`~ ~Nb@"   
$8N$9BNP**~``~~ D4$DEBB@@AN_C#@2#gB$L       ]@@@L 
5QdQ#$^"-       B$g@iBQ@@B@@@y@@@_4F        $@BF  
R7@g6M8y-$& q   $@@@@@N@@@@@@@@@@M@U      ,DP"    
35@N#4Rg2#z&y@  \@@MNR$]@@@@@NN@@N@L,    '        
33Kx$R#C%%<&qV/ d@6  dNBN@@@NF .%N@@L             
x&:}` =)'!*   @P   4@  #@@    *dN@&L            
%kxj[K!    -'  B@   B@    N@_    $ `*@y           
/~'9/ '    _ydLB^   #[p.   ]Br       3@E     '"   
  JS      *   ][   jP~~~*  B5'        `@r     *   
s ^  ^'"     d@E  y@F    .#F           @.         
^u*      _.gy@@@WN@@yg,g#@AWg_d@@&.__  N#         
^      .gB@@@@@@@@@@@@@@@NBB@@@@@@@@@@@@@@y.      

cm.txt

This isn't an image, it is just text (try cutting & pasting it). Unfortunately, your browser may not be able to find the font I used (if it doesn't look exactly like the next image, it isn't using the same font), so it might not look as good as possible
Image using text characters


cm-txt.jpg

This is an image (of text), and is using the "right" font (so this is a good as the text rendition gets, even thought it isn't all that great). This image is not really all that useful in and of itself, since if we are going to have an image, why not use the original, right? It is, however, useful in order to produce the next image.
Morph


cm-morph.gif

Morphing is a standard technique for converting one image into another over a sequence of frames. It produces neat effects.

Utility

As I mentioned above, there is nothing particularily useful about the above technology - it is just a silly little amusement. If I really stretch things, I can make up some really contrived and lame-ass benefits:

  1. Can send the text version of images as signatures in emails

  2. I could create some lame-ass "artists statement" talking about how the morphing animations are "art". For example:

    My work represents an introspective journey into the nature of sight and perception. Here, in "Fade to White", the viewer undergoes a parallel metamorphosis with my work. Loss of vision is an ever present fear to most artists. What would I do if my sight to start to fade. Or be lost altogether?

    Geesh. Artist Statements are amusing in their pomposity (kudos to Christine for helping up the pomposity level). Even if the underlying statement is valid; I have often wondered how I could cope. Programming would be a bitch without sight. And I am kinda concerned about my sight nowadays. I have a tiny little black floaty thing in the vision field of my left eye that is kinda disconcerting. We need replaceable eyeballs! Soon!

  3. The poor quality of the text versions of images can actually be a benefit in some situations. For example, it could be used as an initial "work safe" version of risque images hidden behind a link. Or used as a "secret" placeholder that can be publicly posted yet which means something much more significant to people who know about the underlying original image. For example, the following text representation means more to a few people on my LJ than it will to others ;-) Hee hee.
    @@@@@@@@@@@@@@@@@@@Bgg_.                                          
    @@@@@@@@@@@@NNN@N@@@@@RNNyg_                             ,y14AgwmW
    @@@@@B?%7*%@E/'"-`^~R@@@@@@@Nby&.._             __._.ng#N*"   '`~'
    @@@@@@bL'  ^Ny-       '"?#@N@@@B@1^        jag@@N$5^              
    @@@@@@$p*    "a&._              "4          `^`                   
    @@@@@@@@1                                                         
    @@@@@@@@[<                                                        
    @@@@@@@@NZ+                    -                                  
    @@@@@@@@@@Dxg                   '                                -
    @@@@@@@@@@@M%            .+                                      B
    @@@@@@@@@@@@5#r_    _   '}    ,                                %N@
    @@@@@@@@@@@@@@{my4aea6~'gk                       -          _gM@@@
    @@@@@@@@@@@@@@@RVK9aAb1QBA                        ^,.      yd@@@@@
    @@@@@@@@@@@@@@@@@B%@@@@@B@$x_                    s]Ngg8@yg@@@@@@@@
    "J"PF*?*MM5NNN@@@@@@@@@NB@@@@Ng_                73%"&y@@@@@@@@@@@@
                 `  "7?@@@@@@@@BN@@&-            .k,Jg@@@@@@@@@@@@@@@@
                       ~'KN@N@@@@@@@@Ngg._gggm4#00zyN@NMP$%5P5?PVN##@@
                        K0A/^jDN@@@@@@@#"   '`"7)(>v& "          '    
              -  ~     `\.-J" w+"KK35B     '[, %*/_'>"                
                '_     .~   ^ ^h!3&A@I     >)s%\7 -<-                 
     '         ^ .      '''L>     ^>#h-    /6i\       ~  -            
          ^     =    ~ '  ^       '}']'   ']<- '       '      _       
             ^ _ ,     ^             `    ~;  _,                '' _  
    

    lun.txt

    I really cannot describe how amusing I find the above. Or how amusing the idea of posting a morphing userpic based on it is.

Future Work

Of course, this is only the tip of the iceberg, with respect to finding the best possible text representation. Here are some ideas for future improvement:

  1. Use a unicode font that has more characters available. The font I'm using now only has 93 printable characters. The more characters available, the better the "best match" will be.
  2. Measure how good the image-to-text conversion is for one font, then repeat the process for any number of additional fonts. Pick the font (and corresponding text output) that maxizes overall similarity, not just individual rectangle-to-character similarity.
  3. Although fixed-width fonts make things easier, with a little more work we could also get the program to handle variable-width fonts, and I suspect that they might provide even better matching potential.

Questions

  1. Do you have any images you want textified?

  2. Any benefits of this silliness that I haven't thought of?

  3. Any HTML experts out there happen to know how to force browsers to use a specific unix-style font via CSS? I'd like to specify this font:

    -misc-fixed-medium-r-semicondensed--13-120-75-75-c-60-iso8859-1
    6x13). The closest I've been able to come is:
    color: black; 
    background-color: white; 
    font-family: "Courier New"; 
    font-size: x-small;
    

    but the above isn't a perfect match (the 6x13 font is fixed-width sans-serif, whereas the above Courier New font is serifed. And the sizes aren't exactly right either.

    There is the secondary problem of LJ ignoring <style>...</style> specifications (and maybe even style attributes within tags, although I haven't verified that). Which means that although the above CSS gets fairly close to the 6x13 font, I cannot specify the font in LJ. Silly LJ peeps; why they inhibiting us?

  4. Did you find the (many) hidden link(s) to amusingly explicit naughtiness?

index interests guestbook filter random 1:C 2:lovelang 3:johari
LinkReply

Comments:
[User Picture]From: imarvintpa
2007-05-11 07:52 pm (UTC)

I hate to burst your bubble...

This has been around for a very long time called ASCII Art.

But it is definitely fun to try something "new to you" out.

IMarv
(Reply) (Thread)
[User Picture]From: metawade
2007-05-11 08:09 pm (UTC)

Re: I hate to burst your bubble...

Hi IMarv. No worries, you haven't burst my bubble at all. I'm quite familiar with ascii art, and also knew there were Image to ASCII converters. As I mentioned at the beginning of the entry, I wasn't making any claims of originality.

The image experimentation was just a means of gaining some familiarity with the Image::Magick perl library. I am quite enthused about targetting browser-rendered text, because one has access to a huge number of fonts and specialized unicode characters. Especially when combined with variable-width font support, this has all sorts of potential, although admittedly the search space becomes ... large. Such tedium is what spare cycles are made for though :-)

You happen to have any suggestions wrt Q#3?

(Reply) (Parent) (Thread)
[User Picture]From: imarvintpa
2007-05-11 08:25 pm (UTC)

Re: I hate to burst your bubble...

If you want to make just HTML based imaging, the gosh-darn-it technique is to make 1x1 pixel sized somethings and color code them. It would be an impressively data explosive technique, but fully portable in an insane way.
Next up from that is pick a fully fleshed out unicode font. Render each character at a fixed font size and take their resulting B&W bitmap and make it a bit stream. Record it into a database. If you're lucky, you picked a font with a fixed font space. Then you just have to break up your image into smaller images that match the size of a character in the font. You can do a search of that specific bitstream for each character and end up with the same image. Failing that, it'll take fuzzy searching to find characters with the most matching bits to the block in question.
Assuming an 8x16 font size, you are trying to find the character that has the closest count of matching bits to 128. You might want to check into spell-checker tools, it seems somehow related.

CSS:
color: black;
background-color: white;
font-family: "-misc-fixed-medium-r-semicondensed--13-120-75-75-c-60-iso8859-1", "Courier New";
font-size: x-small;

Although, there might be a "friendly" name your browser/font system uses for the font and that may be needed instead. BTW, if a browser doesn't find a font, it goes to the next one in the list.
(Reply) (Parent) (Thread)
[User Picture]From: metawade
2007-05-11 08:41 pm (UTC)

Re: I hate to burst your bubble...

Hah. The 1x1 pixel idea kinda takes the fun out of things :-)

Note entirely sure why you are re-explaining my own algorithm back to me, but maybe I'm missing something. The discussion in this entry wasn't just a vapor-ware exercise in "this could be done", it was an assertion that it already has been done. The bit-stream database, the use of string-similarity technology (I started with a simple XOR and count-the-1's strategy, but the dynamic programming string-similarity method provides better matches sometimes). The next step will be to expand to support variable-width fonts, differing font weights, and multiple fonts at the same time. I'm puzzled by your paragraph. Why are you teaching your grandma to suck eggs?

The CSS *is* useful, however. I hadn't realized one could just embed an X-style font specification into CSS. I assume, however, that this will only work on a machine that is running unix/linux/cygwin, and thus isn't a useful strategy for showing other people a text-image?

(Reply) (Parent) (Thread)
[User Picture]From: imarvintpa
2007-05-11 08:48 pm (UTC)

Re: I hate to burst your bubble...

My re-explaining your algorithm back was my fault. Not all of my comprehension ran in the right order. Good new is, we're on the same page!

For the CSS, I can't test its validity on a *nix system, but it is my best guess at something that would work. Just need to find its closest relative in fonts in the windows world and add its name to the list. I'm not much help there either.

IMarv
(Reply) (Parent) (Thread)
[User Picture]From: tisiphone
2007-05-12 12:49 am (UTC)
Something is twinging in the back of my mind about applications of this to steganography. It'd be pretty transparent since you just laid it all out there, but it's still kind of a cool thought. (And if you use <code> you get a fixed-width sans serif font. I'm pretty sure, anyhow. I know it's fixed width. I'm too lazy to find out if it's sans serif. It might actually be Courier New, come to think of it (have you tried plain Courier?))
(Reply) (Thread)
[User Picture]From: metawade
2007-05-13 03:32 am (UTC)

<code> apparently maps to "Courier New" usually, which sadly is serifed. And I'm pretty sure Courier was too, but not I'm not sure, so I'll experiment. I have lots of plans for improvements - hopefully a full-fledged unicode font (and the use of variable-width fonts) will produce much better matches).

(Reply) (Parent) (Thread)
[User Picture]From: luighseach
2007-05-12 08:22 pm (UTC)
Yay, you're back! I have missed you!
(Reply) (Thread)
[User Picture]From: metawade
2007-05-13 03:38 am (UTC)

Maren! I'm totally looking forward to catching up on your LJ, but alas, I won't really be back until at least 8 days from now, and I *really* have to not get too wrapped up in LJ again until I get myself back on-track again. But yes, the writing muse has been visiting me, so I'll be LJ-present again sometime soon.

(Reply) (Parent) (Thread)
From: iisz
2007-05-12 09:45 pm (UTC)
Oh geez, the drama. Look, just because the girl likes the horse dudes doesn't mean she wants hay for her bithday. Just apologise and get your own wierd axey thingy.

*pretending I know what the hell you were talking about* :)
(Reply) (Thread)
[User Picture]From: metawade
2007-05-13 03:35 am (UTC)

Totally made me laugh. Human mouths just aren't designed for hay.

And it is called a halberd.

(Reply) (Parent) (Thread)
[User Picture]From: crittershay
2007-05-15 05:47 am (UTC)

Too much time on your hands....

KIDDING... I remember when plays in my head... kool stuff... :P

(Reply) (Thread)