Loading
- Find Search Engine Rankings... via the Command Line
- Just as Mario: Using the Plan9 plumber utility
- Extensibility in the Acme text editor
- Creating New Text Objects in Evil-Mode (Vim emulation layer in emacs)
- An introduction to the magic of Google Tag Manager
- Version Control: Started using git and github (and how to set-up a remote git server)
- Editor Magic: emacs, vim, acme and the return key
- Polishing a Shoe
- Adding Notes to Emails in Gnus (in Summary View)
- (Why) Juggling for Programmers (and other computer people)
- Back to gnus (emacs mail reader)
- Timeline of whatlanguageis.com: first Django project
- Language Detection in Python with NLTK Stopwords
- It's Not Only the Politicians: This country is completely fucked up
- Aprende a recordarlo todo: el método del palacio de la memoria
- The Mind Palace Memory Technique (or: what I'm watching on TV lately)
- Learning to use vim in my iPad (even if I'm an emacs geek)
- Good Books I've Read in 2011: Perfect Gifts for Geeks!
- Working on the go with an iPad, a Bluetooth keyboard and a 6sync account
- Again and Again: Playing Go
- Time Tracking Like There Is No Tomorrow (with Javascript and HTML5)
- Another World for iPad and iPhone Review
- The Mental Drawer: Never miss an idea again
- Using Gephi with Google Analytics to visualize keywords and landing pages
- Breaking Blogging Conventions
_
Via pixelfrenzy@flickr
BEWARE! THE SOFTWARE DESCRIBED HERE IS JUST FOR PERSONAL AND VERY_ LIGHT USE. ITS USE BEYOND PURELY RECREATIONAL VALUE IS AGAINST GOOGLE SEARCH TERMS OF SERVICE, AND I DONT WANT YOU OR ANYONE TO STEP THAT LINE. ANY USE OF THIS CODE IS AT YOUR OWN RISK.
Well, after this scary paragraph, lets get to the real meat. Which boils down to just a few lines of bash.
Im a command line interface geek. I enjoy tools like taskwarrior or imagemagick. And every time I have to chew some data, I go down the command line route. In the last months I have found the incredibly effectiveness of awk, sed, head, tail and tr to convert what some web service spits into useful data I can feed to R to do something useful.
Today I was thinking about search rankings, and wondering if I could write a small tool that eased my occasional curiosity. Yes, I have done this by hand: search for some keyword and look through 8 pages of results looking for one of my websites. I dont do it very often, but sometimes it has to be done. And since I love automating things and the command line, I wrote the following blurb. I dont claim it is beautiful, but it works:
#/bin/sh
# Perform a web search in Google via the command line. Usage:
# automatic.sh "search term" domain pages
# domain stands for what goes after the dot in "google." and
# defaults to com. Pages is the number of search pages to
# process. Defaults to 1, since this script fetches # 100 results per page.
domain=${2-com} pages=${3-4} span=$(seq 0 1 $pages) for num in $span do iter=$((num*100))
# This sets the user agent to Lynx, a command-line web browser to # let google know its better if there is no javascript and fluff # laying around. wget --header="User-Agent: Lynx/2.6 libwww-FM/2.14" "http://www.google.$domain/search?q=$1&start=$iter&num=100" -O search$num -q # Comments about this piping: # The sed Extended command looks for patterns like # href="/url=something", with the goal of grabbing that something: # thats the result URL from the search. It captures this group # and rewrites it as "SearchData, something" with several new lines # for readability (in case you remove the grep pipe.) # The grep is just used to prune all lines that are not search # results, and then awk is used to print the search result number # and the URL sed -E "s/<[^h]*href=\"\/url[^=]*=([^&]*)&[^\"]*\">/\n SearchData, \1 \n\n/g" search$num | grep "SearchData" | awk "{print $iter+NR \" \" \$2}" done
# The script leaves a lot of files named "search??" laying in the directory it # is executed. In some sense this is a feature, not a bug. You can # then do something with them. If you dont want them laying around, # add rm search$num before done.
To find the search standings of a particular page for a particular term, just pipe through grep or ag:
./automatic.sh "learn to remember everything" | grep "mostlymaths"
For extra fancy points you can use this to create reports inside Acme, if you are so oriented.NOTE: Its best to open the videos in full screen. Also I have added a few line breaks or readability in the code snippets that will make them not work correctly. Its not hard to find where they are, if you run into any problems let me know.
If youve been following this blog, youll know Ive been using Acme and related Plan 9 from User Space utilities lately. One of its pieces is the plumber. And knowing what it does may be tricky:
Plumbing is a new mechanism for inter-process communication in Plan 9, specifically the passing of messages between interactive programs as part of the user interface. Although plumbing shares some properties with familiar notions such as cut and paste, it offers a more general data exchange mechanism without imposing a particular user interface.
From Plumbing and other utilities by Rob Pike
WHAT IS THE PLUMBER?
Its somewhat hard to explain at first. Ill try to give you a simple example from Mac OS. In Mac OS you can find a command line utility named "open" (/usr/bin/open). From its manual entry you can read:
open -- open files and directoriesSimple and easy. And it just does what youd expect: * open something.jpeg will open this image in Preview (or its default opener) * open afile.txt will open this text file with its default opener, too. * open http://web.something will open the webpage in the default browser. I think you get the idea. Open just opens something. If you use the command line in Mac OS, you have probably used it on and off to quickly open images or pdfs. I know I have. The Plan9 plumber is like open on steroids. Why? PLUMBING SUPERPOWERS The strength of the plumber is two sided: * Completely configurable * System wide (kind of) Completely Configurable The plumber works with rules, and you can just write your own cool roles in a file ~/lib/plumbing What does a rule look like? Here is a rule to open an image using the Mac OS open utility: type is text data matches [a−zA−Z0−9_–./]+ data matches ([a-zA-Z¡-￿0-9_\-./]+) \.(jpe?g|JPE?G|gif|GIF|tiff?|TIFF?|ppm|bit|png|PNG) arg isfile $0 plumb start open $file Rules work in a trigger-and-fire way. A set of tests make the trigger, if the text you pass to the plumber matches, the rule is fired. In this case: * The type of the data has to be text (in fact is the only type the plumber handles) * The content of the message has to match this regular expression [1] * The content of the message has to match this regular expression, with groupings * The argument group $0 (the whole regexp match) has to be a file, then the variable $file holds its full path * THEN we run "open $file" [1] This is needed. From the plumb manual section 7 (man 7 plumb or right click plumb(7) inside acme):
The first expression extracts the largest subset of the data around the click that contains file name characters; the second sees if it ends with, for example, .jpeg. If only the second pattern were present, a piece of text horse.gift could be misinterpreted as an image file named horse.gif.This relatively simple example is big enough to allow us to make pretty cool things with the plumber. Although this example only allows us to type plumb filename.jpg and the image will be opened via "open". But we may get as fancy as we like, for example: # isbn10 search through Amazon type is text data matches ([0-9][0-9][0-9][0-9][0-9][0-9] [0-9][0-9][0-9][0-9]) plumb start open http://www.amazon.com/s/?field-keywords=$1 * Type has to be text, as always * Match exactly 10 digits (Plan9 regexes dont have brace count) * Then search the ISBN through Amazon Now we can run plumb 0836213122 to find what book this ISBN points to. Neat, isnt it? Well, it just gets better. SYSTEM WIDENESS OF THE PLUMBER When I say the plumber works system wide, I mean that in the Plan9 ecosystem (or in plan9ports) anything the plumber recognises as "special" can be easily plumbed. For example, Im typing this in acme, thus I can right-click this ISBN (0836213122) to switch to Chrome and find what this book is. And if Im in a 9term window, I can middle click-and-release to get the same. Its neat, but of course it gets neater: right clicking in an expression filename:lineno gets you to this match, making grep a whole lot more useful. For me, its main strength is inside acme, since it is almost the only piece of Plan9 I use with regularity (I use zsh which is pretty good as it is.) As such, using the plumber we can add even more awesomeness to this editor: type is text data matches ag:"?([a-zA-Z¡-￿0-9_\-./]+)"? arg isdir . plumb start zsh -c ag --nogroup --nocolor --search-files "$1" $dir | plumb -i -d edit -a action=showdata filename=$dir/ag/$1 This lets me write ag:word or ag:"something longer" to search for word or "something longer" recursively in the current working directory using The Silver Searcher. _The Silver Searcher _is a source code file searcher, similar to ack. Ack and tss are both faster than grep because they omit .ignore, backup and non-source files. Of course this can be overriden, but since I usually work only with source when using grep, it is a great tradeoff. The --search-files is a hidden option, without it line numbers are not rendered when ag is ran from another program (see this github issue). Or I can add the following to use my wikipedia quick searcher script (based on this neat commandlinefu hack): type is text data matches def:"?([a-zA-Z¡-￿0-9_\-./]+)"? plumb start zsh -c wikiCLI.sh $1 | plumb -i -d edit -a action=showdata filename=$dir/wikipedia/$1 So I can write and use def:europium in case of need. Of course, it is far more handy to use acmes 1-2 chords for this: select europium with button 1, select wikiCLI.sh (the name of my script) with button 2 and without releasing, tap button 1... As you can guess, this wont work in a 1-button Mac laptop without external mouse, since you cant press button 1 while pressing button 2... After all, button 2 is button 1 while pressing Alt. To make it work, I patched devdraw in plan9ports: diff -r 1bd8b25173d5 src/cmd/devdraw/cocoa-screen.m --- a/src/cmd/devdraw/cocoa-screen.m Tue Mar 19 14:36:50 2013 -0400 +++ b/src/cmd/devdraw/cocoa-screen.m Sat Apr 06 20:02:44 2013 +0200 @@ -847,7 +847,9 @@ case NSFlagsChanged: if(in.mbuttons || in.kbuttons){ in.kbuttons = 0; - if(m & NSAlternateKeyMask) + if(m & NSControlKeyMask) + in.kbuttons |= 1; + if(m & NSAlternateKeyMask) in.kbuttons |= 2; if(m & NSCommandKeyMask) in.kbuttons |= 4; And now you can press Ctrl while having Alt pressed to send a "button 1" chord. Simple and innocuous patch. FINAL REMARKS As you can see, the plumber can be useful in every day tasks, enhancing acme. Together with the ability to execute text, the plumber makes acme and incredibly different text editor. Its not vim, its not emacs. It is acme, in its own right a very powerful piece of software. I hope these posts Im writing lately are making you have an itch to use acme and have a look at what Plan 9 has to offer.
Text editors. You hate them or love them. Praise them with religious zeal, and attack them with the same power. Ive been an emacs user for the last 8 years, getting as deep as I could without checking the source. And the past few months I have started using evil-mode in emacs, to get some taste of vim in my daily editing (mostly text objects.)
This is how this post looks like. You can see an adict window by the side
Theres still a third contestant in editor-land, for me. It is Acme, the odd editor from Plan9 from Outer Space, the even-more-odd operating system from Bell Labs. Theres no need to install Plan9 and fight against your current hardware. If you are in any kind of Unix derivative (Mac OS, Linux) you can install Plan9 from User Space, a port of most of Plan9 to work in user space (as you may guess.) Plan9 is a whole different thing from other Unix systems, and Acme is an incredibly different beast from any other editor you know.
I can start with a screenshot of it:
This is how this post looks like. You can see an adict window by the side
This is Acme. I hope you like this shade of yellow and this shade of blue. Theres no way to change it without getting into the source code and recompiling. It may be sound odd, but I kind of like it. Its refreshing. In emacs and vim it is very easy to get a beautiful colour scheme (I use solarized-dark everywhere I can,) but this means you can choose. And choosing means a decision, with pros, cons and whatever. Just screw it and pick blue and yellow.
Once you are used to it, you have to face something "worse." If you come from emacs or vim this will sound just horrible. Wait for it. Everything is done with the mouse. Yes, you read that well. No keyboard shortcuts (well, there are a few, Ill get into these in a short while.) Mouse clicking, moving and chording. The likes. I know this will sound stupid, a waste of time, prone to carpal tunnel syndrome. Let me go on for a while.
Emacs and vim users alike like to bash any other editor in the grounds of speed. I can refactor faster than you can, is almost the motto. Watch how fast I type, thus how fast I change code. Im one of these, I usually dont even have to think when Im doing "something" in emacs or vim and changing stuff. But then again, how often Im changing stuff?
Emacs and vim make easy changing whats there. Multiple-marks, text objects, quick jumps. All this is there just to make changing stuff fast. Agree? Ok, go on. If you dont, no problem. Go on anyway.
Now the revelation: most of the time Im writing, Im creating new stuff, not rewriting or moving old stuff. Shocking? Watch your own coding/writing habits. Yes, I change whats in text strings (ci", in vim) and is incredibly fast. In Acme, you can double-click after the first quotation mark (or just before the last) to select everything inside a pair of delimiters (a pity it is not smart enough to understand dollar-delimiters as used by LaTeX.) But the point is not that speed. What are you changing this string for? Did you wait to think about it or you just changed it, compiled it, checked it and went back to square one?
PAUSE: THE FILE SERVERS
Acme and Plan9 follow a special philosophy: in some sense, everything is a file. And most programs (I could say all, but Im not that into Plan9 to be sure) act as file servers. Acme is just one of these: everything you can see in an Acme session is a file. For example, this text Im writing (you saw it in the previous screenshot) has window ID 10. So...
acme/10/ is the directory associated to this text acme/10/addr is a file with an address position for text insertions acme/10/body is a file with the contents of the editing window (cant overwrite) acme/10/ctl is a "file" (socket-like) that allows you to send commands to the window acme/10/data is a file with the data of the editing window (can overwrite) acme/10/errors is a file with data spat by commands executed by this window acme/10/event is a "file" (socket-like) where you can read/write the editing session acme/10/tag is a file holding the contents of the tag (the menu above) acme/10/xdata is a file with the data (addr bound) of the editing window
What does this mean? It means I can write code in any language that can manipulate this text. Read this out loud: I script something, and make it work with the text Im editing. Think about indenting, linting, type-checking, done against the working copy, not the real file. With output to a special buffer associated to the file. Extending the editor is just a matter of writing a program.
BACK INTO ACME
Back into Acme. In Acme theres no GUI: text is the user interface. TUI. Every window is composed of two pieces: the text in the window (its "body") and a tag above it (in blue.) If you want to copy something, select it and middle-click on Snarf. Then put your cursor where you want to paste and middle-click on Paste. Done. Of course, Snarf and Paste can be anywhere. In the tag menu, in the text you are editing or even in another document. They are just words. Words that do the work.
But the same works with shell commands. I can type date and middle-click it, to get the current date in a new buffer. Same goes with ls. Or even |md5sum to calculate the checksum of some text. Or append something to a window. For example, theres an easy way to make small queries to Wikipedia via the command line (see http://www.commandlinefu.com/commands/view/2829/query-wikipedia-via-console-over-dns) I wrote a script to do it, sitting in my path, so I can now type here
Result:
DIRTY, CLEAN, PUT
"Acme (\; , the peak, zenith, prime) denotes the best of something. Acme or ACME may also refer to: Acme Corporation, a fictional company in the cartoon world of Looney Tunes, ACME Detective Agency, a fictional detective agency from the Carmen Sandiego seri" "es of computer games and television shows, Acme (album), the sixth album by the Jon Spencer Blues Explosion, Acme Novelty... http://en.wikipedia.org/wiki/Acme"ON BUTTON CLICKING
I use a Macbook (almost 5 years old already, and still kicking.) And as you may guess, it only has one button. So, how do I manage to use middle and right clicking with ease? Well, easy. Or almost. Pressing alt while clicking simulates middle click, command while clicking simulates right clicking (in Acme, not in general.) Easy, since alt is in the middle and command just right to it.
Problem is, chording is most awesome with a real 3-button mouse. Why? Id rather search for wikiCLI.sh Acme (text editor) to get:
;; Truncated, retrying in TCP mode. "Acme is a text editor and graphical shell from the Plan 9 from Bell Labs operating system, designed and implemented by Rob Pike. It can use the sam command language. The design of the interface was influenced by Oberon. It is different from other editing " "environments in that it acts as a 9P server. A distinctive element of the user interface is mouse chording... http://en.wikipedia.org/wiki/Acme_(text_editor)"
Doing so is a little more troublesome: select "Acme (text editor)" and select the command with the second button (to execute) finally click: clicking sends the last selection as argument 1 to the program. Doing so with a Mac trackpad is impossible: theres no way to simulate a left-click while middle-clicking. I also find theres a glitch here: theres no way to redirect the output of the command. Its either overwriting the selection of the argument, or goes to the +Errors window.
SELECTIONS, REGEXES AND OTHER FURRY ANIMALS
How can I select everything? :0,$ and right-click. Done. Want to replace all instances of acme for Acme? Middle click this: Edit ,s,acme,Acme,g The sam syntax is easy but... odd. The first , is to select everything, s to replace acme for Acme, g for global. Easy? Not much, but Acme is just different. And works. Edit is the command to execute an editing command, by the way.
You can also do regex searches. Like :/interface/ or :/click[i|.]/. And you can get fancy, by doing filename:. For example, acmescripting:/interface/ in another window would open acmescripting in the first instance of interface. And acmescripting:20 opens it, selecting line 20. As you can see, filename is implied to be current file in case of doubt. Also, this kind of referencing works nice with most compilers and linters.
KEYBOARD SHORTCUTS
There are a few keyboard shortcuts, even if Acme is mouse-centric. Press Esc, and all text written since the last click is selected. In addition to this, we have the other standards:
C-U –> Delete from cursor to start of line.
C-W –> Delete word before the cursor.
C-H –> Delete character before the cursor.
C-A –> Move cursor to start of the line.
C-E –> Move cursor to end of the line.
Nothing more, nothing less. Minimal, isnt it?
SCRIPTING POWER
Finally, I want to show some scripting power of acme. I introduced the concept of the filesystem a few paragraphs ago. Now, lets see how it can be used. Lets say for example Im an avid C programmer, and like to have code neatly indented. Well, an option is to write a script that uses the indent command line program to indent the text in the window. How? Now comes a trivial example, not written in the best way possible
#!/bin/zsh #9indent echo "WinId is: " $winid echo -n "1,$" | 9p write acme/$winid/addr echo "Selected whole contents for overwriting with write" 9p read acme/$winid/body | indent -st | 9p write acme/$winid/data
First is to check which is the ID of the window. When invoking a command, its environment has it in a variable, aptly named winid. To overwrite the contents of the file, we set the addr to the whole file with the selector 1,$. We do so by piping to 9p with the command write. 9p is the middleman, allowing us to read and write files in 9P servers, like the ones acme and other Plan9 programs offer. Finally we get the body, indent it (-st is the command to use stdin in indent) and pipe it to data. Done! This indents a well-formed C file as expected.
Add this file to your path and add 9indent to your tag. Ready to indent by middle-clicking.
A slightly more complex example is to generate the output of a Markdown file. The code is as follows:
#!/bin/zsh echo "WinId is: " $winid format=$(9p read acme/$winid/tag) echo "Tag is " $format echo "Format in tag" case $format in *"latex"* ) echo "latex ouput selected" format="latex" ;; *"groff-mm"* ) echo "groff-mm output selected" format="groff-mm" ;; *"odf"* ) echo "odf ouput selected" format="odf" ;; *"html"* ) echo "html output selected" format="html" ;; * ) echo "Unrecognized format, defaulting to html" format="html" ;; esac echo -n "1,$" | 9p write acme/$winid/addr echo "Selected whole contents for overwriting with write" 9p read acme/$winid/body | peg-markdown --to=$format | 9p write acme/new/body echo "Wrote the html-markdowned version to a new buffer" last=$(9p ls acme | sort -g | tail -n 1) echo "Get last created buffer" echo -n "clean" | 9p write acme/$last/ctl echo -n "0,0" | 9p write acme/$last/addr echo -n "dot=addr" | 9p write acme/$last/ctl echo -n "show" | 9p write acme/$last/ctl echo "Moved to beginning"
This is slightly more complex, at least on the shell side. It checks the tag for one of the accepted formats for peg-markdown and then creates the formatted output in a new window, by writing to acme/new/body. Then I want the cursor to be at the beginning of this file, not at the end (as is the default.) It was slightly tricky, but the best way was to sort in numerical order and get the last-created window (thats this tail -n 1) then to set the address at 0,0 and set the dot (selection) at address by writing at the control file. Then the command show makes the window show the selected position: 0,0. Done! Intersped among all this is a "clean" command, to make this new window to close.
Here you can see a video of these scripts in a sample use (and youll see how I miss a middle click - execute - for a right-click - open)
A window can be dirty or clean. It is clean when the contents and the disk file are the same. It is dirty when it is being edited. The best way to know if it is dirty is if you see "Put" in your tag menu, just beside the vertical bar. By middle-clicking Put (or Putall in the main tag) you save this file and mark it as clean.
Also, making a window clean makes closing it quicker (middle click in Del.) Dirty windows need to be Put, or you have to Del again.
THATS ALL, FOLKS (FOR NOW)
I have yet to introduce the plumber, a mechanism that allows you to open arbitrary files (using rules) from within acme. For example, I can open pdf files by right-clicking on them (i.e. some.pdf) but instead of using page (the Plan9 image viewer) I use MacOS Preview. I was forced to do so, since page cant handle all the fonts in a LaTeX generated PDF, so for me its useless. Ill probably write how I configured the plumber in my next Acme installment.
In some sense, the plumber is like a system-wide, app-deep "open" mechanism. In Mac OS, you can "open" almost anything from the command line. If you open an URL, your default browser opens it, if you open an image, Preview handles it. Plumbing is like "open 3.0" but it is hard to manage :/
Below you can see another video with a simpler scripting: browsing reddit from the command line, inside acme. The Python code snippet that gets Reddit data is available in this gist: reddi.py
The links to Practical Vim are affiliate links to Amazon. Beware!
So... last January I was in a flight to London, preparing for an intense, 12 days course on traditional shoemaking (English hand-welted shoes, improving our knowledge at The Fancy Puffin.) And my flight read was Practical Vim. Most of my readers are already aware Im an emacs guy, so the main question is WHY?
I love knowing many tools. Programming in many languages, knowing enough about any thing. I couldnt stand all these people saying Vim was superior (even though most had never touched emacs,) and since knowledge is power, I thought a little more (for me) could not hurt (mwahahahaha!) I picked Practical Vim and started reading it a few days before leaving, and finished it during the few non-shoemaking moments of that course.
When I was back to having a computer, I was torn between what I learnt about Vim and looked cool and all those things I already loved about emacs. I have MacVim a few tries, writing short pieces, writing, testing, rewriting. Clearly there was no way I could give up gnus, org-mode (even though Im not using it that much lately) and all my complex configuration options. So... how come? Even though modal editing feels weird, text objects are cool. Being able to "say" di( and get it to delete inside a pair of parentheses is way cool. Basically this is what sold me as a great feature.
I decided to use evil-mode in emacs. Best of both worlds: I still had all my emacs tools AND text objects and some modal editing. I missed having numbered lines, although I just googled for it... And theres linum-mode (since emacs 22!!) which gives them. But I miss ex commands: being able to move any line anywhere just by issuing :source m destination is powerful. Evil-mode supports some ex commands, just not copy or move. Sigh!
Today was the first day I added evil-mode to my .emacs file, and the day started by working in a LaTeX file. In a LaTeX file, most things are enclosed in pairs of dollars, like _$x=1$_. These dollar delimited expressions are what TeX parses as formulas, and they appear as neat images in your target (usually PDF) file. So, most of the things I need to change, delete or move around are enclosed by dollars. So:
* change whats inside a pair with ci$
* delete the whole formula with da$
* delete just the contents of the formula with di$
* copy the formula with yi$
Useful, isnt it? Problem is, evil-mode does not have $ as a text object! Luckily it offers the tools to make new text objects... Getting it to work was harder than expected, though. I was given several choices of functions I could use to create my text object. The best option seemed to be using evil-regexp-range that allows you to give a regexp for the first delimiter and another one for the second. But all tries I have done resulted in it marking the first occurrence in the buffer. No luck. After trying repeteadly with it, I gave up and used instead evil-inner-object-range, where I had to wrestle with point, mark, excursions and the like. When I managed to get it working I just decided I would not touch it again (even if Im pretty sure the save-excursion calls are redundant, as well as most move and mark commands...) It works, dont touch it.
(defun rb-first-dollar-exclusive (&optional arg)
(interactive)
(save-excursion
(setq rb-temp-first (re-search-backward "\\$" nil t 1))
(message "First %s" rb-temp-first))
(goto-char (+ 1 rb-temp-first)))
(defun rb-second-dollar-exclusive (&optional arg)
(interactive)
(save-excursion
(setq rb-temp-second (re-search-forward "\\$" nil t 1))
(message "Second %s" rb-temp-second))
(goto-char (- rb-temp-second 1)))
(evil-define-text-object evil-inner-dollar (count &optional beg end type)
(evil-inner-object-range count beg end type
#rb-second-dollar-exclusive
#rb-first-dollar-exclusive))
(define-key evil-inner-text-objects-map "$" evil-inner-dollar)
(defun rb-first-dollar-inclusive (&optional arg)
(interactive)
(save-excursion
(setq rb-temp-first (re-search-backward "\\$" nil t 2))
(message "First %s" rb-temp-first))
(goto-char rb-temp-first))
(defun rb-second-dollar-inclusive (&optional arg)
(interactive)
(save-excursion
(setq rb-temp-second (re-search-forward "\\$" nil t 1))
(message "Second %s" rb-temp-second))
(goto-char rb-temp-second))
(evil-define-text-object evil-outer-dollar (count &optional beg end type)
(evil-inner-object-range count beg end type
#rb-second-dollar-inclusive
#rb-first-dollar-inclusive))
(define-key evil-outer-text-objects-map "$" evil-outer-dollar)
(Blogger is killing indentation, damn) Enjoy, though!
_If you want to read a quick overview of this in Spanish, check my post about GTM at DoctorMetrics._
Google has just unveiled a new tool: _Google Tag Manager_. I have spent a few hours playing with it (both before and after official release.) And its awesome! Or at least, it has quite a lot of awesomeness, hidden behind a seemingly simple interface.
_Its main feature is the fact that you can get away with just a piece of included code_. You dont need to include your Analytics code on top and your DoubleClick counters in all the pages you need. Theres just one thing you cant do: change the content of your page (via document.write)
SO, WHATS GOOGLE TAG MANAGER GOOD FOR (EXACTLY)?
It centralizes the management of page tagging and tag execution based on rules. A SIMPLE EXAMPLE: you want to track how many people click on the Atom subscribe link in your Subscribe page with event tracking via Google Analytics. This is straightforward with Google Analytics, in fact: you only need to set the correct event for onclick in the page source like this:
_gaq.push(["_trackEvent", "SubscribeRSS", "Sidebar"]);
But what happens when you want to change this tracking category, or add a new tracking to another link? You have to edit all required pages in your site. If the site is well structured, this will be easy, but not all websites are well-structured. _Or you are not the webmaster_. This is the main point: what if you are just an external consultor? EXCHANGING 15 EMAILS TO GET THE IT STAFF TO ADD THIS ONCLICK CAN EASILY DRIVE YOU NUTS (AND COST YOU AND YOUR EMPLOYER MANY HOURS) You dont have to get nuts any more.
After creating a container and adding the GTM code to your site, you just need to add the required page selector in Google Tag Managers rules:
url> contains> "subscribe"
As easy as that... But you also need to add the event tracker to that link. How? The simplest way Ive come up is with some basic jQuery magic. Include jQuery (which you can do for all pages, not just subscribe!) and then set the gaq push event, by selecting the correct link via the href attribute of the link. You can select based on text, or anything you fancy, of course.
$(a[href$="http://www.mostlymaths.net/feeds/posts/default"]).attr("href",
"http://www.mostlymaths.net/p/subscribe.html?utm_source=GTM&utm_medium=Redirect&utm_campaign=Testing")
The options available are almost limitless: enjoy your new GTM goodness! And feel free to contact me for more implementation details or ideas.
The Octocat, Githubs logo
Its been almost 6 years since I used some kind of revision control system. Back then I wasnt sure about which I wanted to use... I settled with RCS, the father of them all. RCS was structurally very simple, with text-based (human-readable) delta files. I liked that. I had all my code and TeX files under revision control, but then I started using more than one computer and it got out of hand very quickly (using RCS or CVS in Windows was quite tricky and had user and encoding problems.) Stopped using it. After a few years I tried darcs... unconviced. Then I put some of my code in the cloud, in Google Code, so I got to use a little SVN. Not for version control, just for having my code online and tidy. Nothing else.
It was about time that I got again into source code control. For part of my work I will use mercurial, so for my side projects I will be using git, to see what else is trendy right now. Yeah, it has taken me ages to do it :D
I put my thesis under control and committed a few revised sections. So far it works wonders with the built-in emacs vc-mode (version control mode.) I even get a neat menu to check in/out or check the commit story. I have not checked yet its interaction with github with push or pull, but I guess it will work as well. If it doesnt, my emacs loving twitter pals Rami and Anti have suggested the magit package. Im not sure about it, because it is recommended when git is your only version control system, and I also want to use mercurial. Only time will tell, though.
By the way, the first real commit I did was for my work in progress with getting some acme magic into emacs. You can check it here in github (which works very well so far... why did it take me so long?).
I also set a private repository in my own server to host my emacs org files and calendar. I followed the following steps at first...
mkdir targetrepo cd !$ git init cd .. git clone --bare targetrepo targetrepo.git mv targetrepo targetrepo.backup git clone targetrepo.gitA somewhat convoluted way to (I think) just doing this, but Im not sure:
mkdir targetrepo.git git init --bareNow you have to connect to it in the local machine (and transfer the local files,) in the desired directory
git remote add origin username@server/path-to/targetrepo.git git push origin masterAnd youre set to go. I got it configured in my server, Macbook and... iPod Touch. Now I have my emacs org-based calendar wherever I go, with (manual) syncing among machines. Reinventing the wheel FTW!
GLENDA, THE PLAN 9 BUNNY. Image copyrighted by Lucent Technologies, hosted by Wikimedia Commons and posted here for information purposes
_OF COURSE I DONT HAVE TO DO THIS, ONE MIDDLE-AGED MAN SAID, CAREFULLY CLEANING THE TABLE WITH A DAMP CLOTH. HE PUT THE CLOTH IN A LITTLE POUCH, SAT DOWN BESIDE HIM. BUT LOOK; THIS TABLES CLEAN. _
_USUALLY, THE MAN SAID. I WORK ON ALIEN - NO OFFENCE - ALIEN RELIGIONS; DIRECTIONAL EMPHASIS IN RELIGIOUS OBSERVANCE; THATS MY SPECIALITY... LIKE WHEN TEMPLES OR GRAVES OR __PRAYERS ALWAYS HAVE TO FACE IN A CERTAIN DIRECTION; THAT SORT OF THING? WELL, I CATALOGUE, EVALUATE, COMPARE; I COME UP WITH THEORIES AND ARGUE WITH COLLEAGUES, HERE AND ELSEWHERE. BUT... THE JOBS NEVER FINISHED; ALWAYS NEW EXAMPLES, AND EVEN THE OLD ONES GET REEVALUATED, AND NEW PEOPLE COME ALONG WITH NEW IDEAS ABOUT WHAT YOU THOUGHT WAS SETTLED... BUT, HE SLAPPED THE TABLE, WHEN YOU CLEAN A TABLE YOU CLEAN A TABLE. YOU FEEL YOUVE DONE SOMETHING. ITS AN ACHIEVEMENT. __THE USE OF WEAPONS_, Iain M. Banks
Maybe you are not aware of it, but Im a shoemaker. Together with my girlfriend we run The Fancy Puffin, a handmade shoe workshop. Go and give it a look, Ill be here when you come back. All our designs are unique and custom made: you can ask for almost anything of us. Checked them? Good! I hope you liked some and are ready to ask for your custom made Oxfords or some fancy woman shoes.
One of the most striking things about good, quality shoes is how they SHINE. Leather (unless its patent leather) is not particularly shiny. That unearthly shine good shoes have in pictures and feet comes from a labor intensive work of polishing.
That shine (which is known as military shine, because US soldiers were supposed to invest a lot of time to have their boots shiny) is relatively straightforward to get, albeit time expensive. Of course the end result is quite striking, as you can see below with my everyday pair of Oxfords. These are the shoes I wear most often (more than 1 day of every 2.) Both shoes have been cleaned of most dust, and both had a layer of horse fat a few days ago (its good to grease your shoes from time to time to keep leather flexible.) One has been polished, the other has not. Can you guess which is which?
I learnt to get that shine from a very interesting blog (The Shoe Snob.) As I said, its not hard. I didnt completely follow the directions of the blog, which probably results in a less impressive (but faster shine.) How did I do it?
1ST STEP: Clean all the dust off your shoes with a shoe brush.
2ND STEP: Pick some cloth (I used rags off an old t-shirt) and wrap it around your index finger. Use it to spread a thin layer of black wax based shoe polish all over your shoe. You can get the layer thinner by repeatedly passing your clothed finger around the same area: if you are very intensive youll get a mild shine. Dont be so intense, but keep polishing until you feel the shine is "almost there". Theres a bit of trial and error here: no problem in getting a little shine in some areas until you get the correct amount of removal.
3RD STEP: get a small amount of water in the lid of your shoe polish (or a small plate.) This step is usually done by spitting on your shoes... Water is cleaner. Get a small amount of polish, put it on the shoe and spread. Then, get the tip of your pinky wet and put a small amount of water in the just waxed area of your shoe. Quickly spread the water with your clothed finger repeatedly. It takes a while for the leather to absorb all the water. The process goes as follows:
* Theres a trail of droplets. You pass your finger on them
* The wax gets wet and turns a dull black. Keep passing your finger in a circular motion around the area
* The wax feels thicker to your finger. Keep at it, slowly passing your finger in a circular motion around the area.
* A shine starts. Keep at it!
4RD STEP: Theres no 4th step: just get the shine all over the shoe. You can repeat step 3 several times, but eventually all the poruses (porus? pori?) of the leather will be clogged with wax and it wont accept any more.
5TH STEP: Actually theres a 5th step. Get some nylon cloth (a woman tights, for example,) wrap it around your finger and pass it all over your shoe. It will get a brighter shine out of it.
And voilà ! A good shine can last many days, of course it will depend on many factors (weather, type of pathways you walk, hours you keep your shoes on and some other.) But a good polished shoe is sure to attract a few looks. It has happened to me countless times!
Sensitive information blanked
THE GNUSNOTES.EL PACKAGE, WITH DELETE AND EDIT CAPABILITIES IS NOW AVAILABLE IN THE MARMALADE PACKAGE REPOSITORY. THIS POST JUST EXPLAINS WHY, HOW AND WHAT (CODE AT THE END, TOO)
Last month I switched email clients (for the 4th time in the past year.) Ive passed through gnus, Thunderbird, mail.app and settled with Sparrow. Just 2 days before Sparrow was bought by Google I left for gnus (lucky moment!) Smaller memory footprint, no harddisk hungry indexes. Works wonders. Would only be happier with a newer Macbook. Oh well.
Im still unsure about going the hardcore way with offlineimap (dovecot, notmuch,) but Im more than happy with the current setup. I can move, delete or archive emails with just a few keystrokes. But Im not in INBOX ZERO yet (wont be ever, I think.) I leave some mails in my inbox when I need to follow up later (to make sure Im on top of things.) Problem is, when you have more than 7 or 8 emails like this you start to lose touch. Furthermore, subject lines sometimes are not very descriptive "Request" is not the best subject. When more than 4 or 5 emails are of this kind, you need to check the email to know what it is about. Losing time. No way!
After all, this is emacs. If you dont like something, or want something, you write it (well, after looking for it.) My goal? Adding notes to email messages. I googled quickly for this with no luck, checked Sachas emacs lisp files just in case. So... Time for some emacs lisping.
You may wonder: dude, use org-capture! This would be a good solution if I looked more often my todo list. But I only look my todo list when, well, I have to do something. Somehow stay-on-top things just distract me in my todo. Id rather have 15 emails in my inbox than 15 more tasks in my already crowded todo list. I still use org-capture with emails, but for long-term catchups or long tasks where I have to do something in the future. When I just have to make sure someone does their work, I leave the email. Works for me, somehow. Getting back to lisp-topic...
Its not my first dive into emacs lisp: Ive written some helper functions to do small things (wrote a wraper to the wonderful mark-multiple package a few weeks ago and Im already using it to mark stuff at point,) I even wrote a major mode to simplify posting to mostlymaths.net, by adding my most usual HTML or CSS idioms with just a few keystrokes. The gnusnotes.el project was harder, though.
First thing was planning how to make it work. The simplest solution: get some kind of email ID, and save it to file tied to a note. I thought for 3 minutes, no need to make it more complex than necessary. Just like a CSV file, I used comma as ID-note separator, newline as ID-note pair separator. Luckily, gnus works because it has the correct pieces. In no time I had found which function determines the current message header data, and among them, message ID (the ID used in backend operations.) Once I had that, I needed a function to find the ID in the file. It was pretty straightforward (find-file and a few search-forward commands.) Something was trickier, though...
Gnus has two main windows: Groups and group summaries. For example, in Groups I have INBOX and sent-201208 as visible folders, I can check all folders (usually hidden) with L. When I open INBOX from here, I go to the INBOX summary, where all unread and marked as important emails show (and this is where my time is usually spent.) I want my notes to appear here, in the summary view. Not inside the email, not disjoint from emails. How!?
Well, for each email a summary line is generated, according to some rules you can set. You can add any detail from the email: headers, references, from, subject, date... And there is also a whole range of custom functions. Adding the flag %u you can choose from a wide range of functions (followed by any letter): In my case, %un calls gnus-user-format-function-n. This function does the lifting: checking the list of notes, for each email.
Problem: even if my notes file is small, hitting it for each email in my inbox can be dreadful. Even worse, the hit would be for any email in any folder. What if I check some archived folder with tens of thousands of emails? Well, these considerations are important, but I just found out that opening a file-buffer for each email somehow broke something within gnus, and filled the buffer with garbage. Later rationalisation turned out these problems :D
The solution (at least the one I used) is to generate a list of cons (msgid . note) each time a group is opened or a note is added. This is done via hooks, tying the loader function to the group preparation hook. Perfect! What else? The function looks for the msgid in this list of notes, and returns the note if successfull (and nothing if unsuccessful.)
To finish, just some cosmetic tweaks: adding some prettyfying of notes via highlight-regexp tied to another hook: the word note appears in italics and with the warning face, and the note appears with the default face. Im not sure how it will look in vanilla emacs, but with solarized-dark I enjoy the colors. Completely tweakable, though.
By the way, my custom summary line is:
(setq-default gnus-summary-line-format "%U%R%z %(%&user-date; %-15,15f %B%s%) %un\n" gnus-user-date-format-alist ((t . "%Y-%m-%d (%a) %H:%M") gnus-thread-sort-functions (gnus-thread-sort-by-date) ) )
This is not yet a neat package with customizable options (yet). Theres even no way to remove or edit notes (except by editing the notes file) since making this work appeared harder than editing this file by hand for the time being... But I may do it some day or another, its not rocket science. Just needs some refactoring of the note-finder. Source code below, with many messages commented for dirty debugging. Enjoy!
;;; gnusnotes.el --- ;; Copyright (C) 2012 Ruben Berenguel <ruben@dreamattic.com> ;; Author: Ruben Berenguel <ruben@dreamattic.com> ;; Keywords: comm, mail ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU General Public License ;; as published by the Free Software Foundation; either version 3 ;; of the License, or (at your option) any later version. ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; You should have received a copy of the GNU General Public License ;; along with this program. If not, see <http://www.gnu.org/licenses/>. ;;; Commentary: ;; Still in beta. Use it at your own risk. Customize the note file ;; location and to install ;; (require gnusnotes) ;; Only tested in emacs 24.1.1 with NoGnus 0.18 ;;; Code: (provide gnusnotes) ;;; gnusnotes.el ends here (defun gnus-user-format-function-n (data) (or (rberenguel/get-gnus-notes (aref data 0)) "") ) (setq gnus-summary-notes "NOTE:") (defun rberenguel/gnus-summary-notes-hl () (interactive) (hi-lock-mode -1) (hi-lock-mode 1) ;(message "Highlighting %s" gnus-summary-notes) (highlight-regexp gnus-summary-notes warning) (highlight-regexp gnus-summary-notes italic) (highlight-regexp (concat gnus-summary-notes "\.\*") default) ) (setq gnus-summary-prepare-hook rberenguel/gnus-summary-notes-hl) (setq gnus-summary-hook rberenguel/gnus-summary-notes-hl) ;(add-hook gnus-summary-generate-hook rberenguel/gnus-summary-notes-hl) (setq rberenguel/gnus-notes-file "~/.gnusnotes") (setq gnus-select-group-hook (lambda () ;; First of all, sort by date. (setq rberenguel/gnus-notes-list (rberenguel/load-gnus-notes)))) (defun rberenguel/load-gnus-notes () (interactive) (set-buffer (find-file rberenguel/gnus-notes-file)) (goto-char 1) (setq moreLines t) (setq rberenguel/gnus-notes-list ()) (while moreLines (beginning-of-line) (setq p1 (point)) (if (search-forward "," nil t) (progn (setq p2 (- (point) 1)) (setq msgid (buffer-substring-no-properties p1 p2)) ;(message "%s" msgid) (end-of-line) (setq p1 (point)) (setq note (buffer-substring-no-properties (+ 1 p2) p1)) ;(message "%s" note) (setq rberenguel/gnus-notes-list (append rberenguel/gnus-notes-list (list (cons msgid note)))) ;(message "%s" rberenguel/gnus-notes-list) ) ) (setq moreLines (= 0 (forward-line 1))) ) (kill-buffer) rberenguel/gnus-notes-list ) (defun rberenguel/get-gnus-notes (msgid) ;(message "Get notes: %s" msgid) (setq gnus-note "") (unless (null rberenguel/gnus-notes-list) (dolist (item rberenguel/gnus-notes-list) ;(message "%s %s %s" item (car item) (cdr item)) (if (equal (car item) (format "%s" msgid)) (progn (setq gnus-note (format "\t NOTE: %s" (cdr item))) (return)) )) gnus-note )) (defun rberenguel/gnus-notes () (interactive) ;(message "%s" (car (gnus-summary-work-articles 1))) (setq rberenguel/gnus-notes-msgid (car (gnus-summary-work-articles 1))) (setq rberenguel/got-message (rberenguel/get-gnus-notes rberenguel/gnus-notes-msgid)) ;(message "Got Message? %s" rberenguel/got-message) (if (or (equal rberenguel/got-message nil) (equal rberenguel/got-message "")) (progn (set-buffer (find-file-noselect rberenguel/gnus-notes-file)) (setq gnus-new-note (read-from-minibuffer "Enter Note: ")) (end-of-buffer) (newline-and-indent) (insert (concat (format "%s" rberenguel/gnus-notes-msgid) "," gnus-new-note)) (save-buffer) (kill-buffer) (rberenguel/load-gnus-notes) (gnus-summary-prepare) ;(gnus-summary-rescan-group) ))) 
From Flickr
No, this is not a tutorial (but I do have written the standard steps to learn below). I began learning to juggle very recently. I just want to answer a possible "why"?
Ive always been fascinated by juggling, and every time I wanted to eat more than 2 small oranges Id try my hand at juggling them. Always followed by bowing and picking them from the ground, of course.
A few weeks ago I was in Dublin for a few days, and in a bookstore I saw a pack of juggling balls (3 balls) and promptly bought them. With them, a small leaflet explained the method, and as soon as I was back home I tried to learn the trick. Ill give you a quick overview:
* Even if I know it sounds lame, START WITH ONE BALL, from hand to hand. You have to get a knack for the height and arch. The throw is not up, but up and to the direction of your other hand. _Every time you throw, think "throw"._
* When you are confortable with one-handed throws, PICK A BALL IN EACH HAND. Throw your first as before, and as soon as the ball reaches its maximum height, throw your other ball as before (and think "throw" as we did before). Repeat this an awful lot. Get the hang for the timing, height and getting used to thinking "throw".
* After a while, you will want to THROW THREE BALLS. Okay. Start with 2 balls in your weak hand (left if you are right handed and right if you are left handed)). Start by throwing one from the left and then one from the right just you did in the previous step. Now, when the first ball lands in your right hand, your second ball should be in the topmost position. Now is the moment to throw the next ball! Dont worry if it falls. Get used to throwing it. Thinking (or saying aloud
* Rinse and repeat... And then get to throw your fourth ball from the right hand. Then your fifth...
_BUT WHATS THE POINT?_ Of course, its fun to watch, and maybe it is fun to do for a while. But, in the long term?
I cant be sure, Im just a data point and Ive been in it for just shy of 3 weeks (on and off, since in July Im doing quite a lot of The Fancy Puffin work and it cuts my "fun" time.) But this is what I found.
First thing I noticed, JUGGLING CLEARS MY MIND pretty effectively. It is not as good as a good meditation session, but while you are juggling (or at least, in the beginning) you cant think of much else. I practised for a few minutes each night
After a few days I noticed something else. MY BACK FELT STRAIGHTER, AND I HAD LESS BACK AND SHOULDER PAIN. When you are juggling you have to keep your back very straight, stand up and move your arms. Something most programmers and people working with computers dont.
I dont know if both these secondary effects will last long. But I thought some other fellow geeks would be interested.
Ive been thinking about getting a new Macbook lately (my heart and wallet were divided among a 11" Air or a 15" Retina Pro). My 4-year old Macbook (Early 2008 I think, 2 GB Ram, Intel Core Duo 2.4 GHz) was showing its age. Mainly when I had the RAM and cache hungry inhabitants of my dock active: Yorufukurou (the best twitter client for managing multiple accounts in a Mac Ive seen) and Sparrow (mail client).
Why was I using Sparrow? Back when I started my current job, I was using gnus in emacs, after the quite successful 30 days in emacs challenge I did in the beginning of that year. But for some reason I cant remember I moved from gnus to Thunderbird, which I could use both in my Mac and my netbook (I have the vague feeling that gnus had some problems in my netbook). I kept using Thunderbird for a long while until I found Sparrow. What did I find so appealing about Sparrow? _Keyboard shortcuts_. Yeah, I want to archive, move and delete mails quickly without moving my mouse. Sparrow had that (Apple Mail and Thunderbird do not.) What baffles me is why I didnt try gnus again. Anyway.
I was so happy, pressing keys and moving emails around. But my Mac was increasingly sluggish: if I had Chrome with more than a handful of tabs, Sparrow and Yorufukurou I could get the spinning beachball of death after doing anything. So you can imagine what PITA was starting MAMP or running a Django test server (for my What Language Is project, among others). The purge command only helped a little, closing Chrome, Adium and others did almost nothing. Rebooting helped some more, but wasnt a decent solution. What was the problem?
Paging. And caching. In the last months Ive been living with around 3-4 GB of free space. It should be plenty for system paging in the disk cache. The problem is that it was not plenty for Sparrows mail cache. I found this out when, after a reboot my hard disk showed almost 3 more free GB. It could not be Chromes cache, since I had cleared it very recently. It could not be my Macs cache, since last time I checked before rebooting I had less than 1 GB paged and had some (little) free memory still. I opened Sparrow and then checked its caches in the Library folder. amounted to more than 1.5 GB, just after booting. Damn.
Just by coincidence around this time I read a post by Sacha Chua, talking with John Wiegley (emacs lisp developer) and thought: maybe its time to give gnus another try. I thought Id need some heavy configuration tweaking or whatever before getting to grips with it. I thought I had a reason for giving up gnus. Just in case aliases or multiple accounts were a problem before, I disabled gmail and only left my work account (Id use gmail in the web interface for personal emails) before starting gnus again. My password was wrong, I changed it. I set up multiple mail aliases for my work account: tweaked the borrowed function I was using to select an SMTP account to switch to my main account in case of unknown account (since I have more than 20 aliases over it and configuring all them was overkill.)
Used gnus again for a while. My Mac was floating around without paging. The fan didnt spin. No hard drive sounds. Quick keyboard shortcuts I almost know by heart by now. Gnus, here I am again.
Last week I wrote about whatlanguageis.com, my first simple try at creating a Django powered site. How did it came to be?
In the last few months have been writing and thinking about several ideas to process social data with Python, undoubtedly motivated by the great book Mining the Social Web. In the back of my mind, I wanted my code to work as a web application (either for me or for others,) and thought that I should comply with everyone else, _learn PHP_ and try to _make Python and PHP be friends_. OF COURSE, THIS IS LIKE GETTING A ESKIMO TO TALK TO A POLYNESIAN. IT MAY WORK, BUT NEEDS A LOT OF SET-UP FROM BOTH SIDES.
This suggests the next question: why me_, PHP? Well, where I work the _de facto_ language for web development is PHP. Im not a developer there, I just do the data analytics and whatever else that needs to be done using the tools I need and just report back. I thought that PHP-ing would help me get it in the workflow. But, even if my mother programming language may be C (although my first words were in GW-Basic,) my languages of choice are _Python_ and _Lisp_ (and I have a great fondness for _Forth_.) I guess you realise how ugly I feel PHP is.
On 29TH MAY I decided enough was enough: if it is something that should almost work as a black box, to automate the part I was already doing with Python code and R data churning, why bother? _Just make it work_. I started reading the free, online Django Book, and I was hooked (you can buy a printed copy in Amazon). It is an awesomely readable book. I had just finished reading Zed Shaws Learn Python the Hard Way to find things I may not know yet (for example, unit testing... silly me) and Django felt just like an extension of what I usually write and what I had read. Views, urls, models, all have this Python feeling. For me, of course, Im not a seasoned Python developer at all. I dont even consider myself as seasoned in any programming language, I just know _I can write code as I need_. But no expert.
I read the first 2 chapters very quickly, and on WEDNESDAY 30 I started testing how Django worked, basic view-url combinations and a test on templating. Just the _hello world_ tests, so to say. I set aside the book while I was thinking what would made a good initial project.
Related to my social media endavours I had a tickling question: HOW DO I DECIDE IF A TWITTER USER DATA IS VALID OR NOT? I.e. if I want to do some kind of language analysis, I NEED TO KNOW THE USER S LANGUAGE. I had a simple _idea_ around that week (I think it was Monday 28,) but didnt get to code it until the 4TH OF JUNE (Monday): you can read about it in Language detection in Python with NLTK and Stopwords.
On TUESDAY 5, after work I created the Django project _whatlanguageis_ in my computer, and read the chapter on _Forms_. Created a very basic stripped HTML (I think I didnt even add a head section...) page where I could enter some text and get a language as a reply from the Django test server. Yay! It was very easy to do: Django does a lot of heavy lifting and this was straightforward to code.
Then on WEDNESDAY 6 I took the day slightly off (only answering to urgent emails and sales enquiries) and wrote the whole site whatlanguageis.com, which involved:
* READING the chapter on Models_ to add a little database of affiliate books, CHECKING how it works
* READING the chapter on the _Admin_ interface to add some books easily, CHECKING it
* READING the chapter on _Deployment_...
* ... which involved a lot of VirtualHost TINKERING with my httpd.conf file
* PURCHASING a domain and setting up the DNS records
* DESIGNING the page (this was the quickest part, Twitter Bootstrap to the rescue)
My initial plan for Wednesday was not of doing everything and not working. My original plan was just to hook the database and check I could read and write the data as expected, forcing me to read the _Models_ chapter. But as soon as I had finished this hooking and checking, the site was almost ready, needing only a neat dressing. Some twitter bootstrap and css-arrows later, the site was working wonderfully in my local machine. Getting it to work in my remote machine was trickier, due to the setup of static files in Django. Its not hard, it is just that it is tricky to find where it fails when it does. In any case, I got it working.
Of course, as I have said a lot of times, whatlanguageis.com currently is just a PROOF OF CONCEPT of two things: _LANGUAGE DETECTION_ and _SETTING UP A DJANGO SITE_. Both worked very well, but now I have to sit down and REFACTOR the whole of it.
The site works well as it is (no need of much refactoring except for the needed changes in _views_), but the language detector needs a proper module, unit tests and a huge revamping. Among others, my FUTURE PLANS FOR WHATLANGUAGEIS.COM include:
* ADDING A LOT OF LANGUAGES. More or less I will reuse the ideas from an old post: The 100 most common words in Icelandic, automatically generated from Wikipedia
* Add the ability to also detect PROGRAMMING LANGUAGES (or at least, the most common)
* Probably add the ability to DETECT LANGUAGE FAMILIES (i.e. when unsure, classify as Scandinavian, or classify as Lisp-like)
* IMPROVE THE RESULTS: Unless the analyzer is completely sure I want to know what it could be (i.e. for very close languages like Norwegian Bokmål and Danish detection is hard, thanks to @qimarc for this specific and useful example)
* Add CHECKS FOR WORD DIVERSITY in the received text
* UNIT-TEST all the things!
I hope you enjoyed whatlanguageis.com tale and liked the memes lying around. Somehow I felt this post needed them. Thanks to memegenerator.net for the ability to create them :)
Lately Ive been coding a little more Python than usual, some twitter API stuff, some data crunching code. The other day I was thinking how I could detect the language a twitter user was writing in. Of course, Im sure there is a library out there that does it... But the NLTK library (the Natural Language Toolkit for Python) does not have any function for this, or at least I was not able to find it after 5 minutes of Google search. So...
I had a simple enough idea to determine it, though. NLTK comes equipped with several stopword lists. A stopword is a very common word in a language, adding no significative information ("the" in English is the prime example. My idea: pick the text, find most common words and compare with stopwords. The language with the most stopwords "wins".
Implementing it was just a matter of a few minutes and around 45 lines.
from nltk.corpus import stopwords def scoreFunction(wholetext): """Get text, find most common words and compare with known stopwords. Return dictionary of values""" # C makes me program like this: create always empty stuff just in case dictiolist={} scorelist={} # These are the available languages with stopwords from NLTK NLTKlanguages=["dutch","finnish","german","italian",
"portuguese","spanish","turkish","danish","english",
"french","hungarian","norwegian","russian","swedish"] # Just in case I add stopword lists
FREElanguages=[""] languages=NLTKlanguages+FREElanguages # Fill the dictionary of languages, to avoid unnecessary function calls for lang in NLTKlanguages: dictiolist[lang]=stopwords.words(lang) # Split all the text in tokens and convert to lowercase. In a # decent version of this, Id also clean the unicode tokens=nltk.tokenize.word_tokenize(wholetext) tokens=[t.lower() for t in tokens] # Determine the frequency distribution of words, looking for the # most common words freq_dist=nltk.FreqDist(tokens) # This is the only interesting piece, and not by much. Pick a # language, and check if each of the 20 most common words is in # the language stopwords. If its there, add 1 to this language # for each word matched. So the maximal score is 20. Why 20? No # specific reason, looks like a good number of words. for lang in languages: scorelist[lang]=0 for word in freq_dist.keys()[0:20]: if word in dictiolist[lang]: scorelist[lang]+=1 return scorelist def whichLanguage(scorelist): """This function just returns the language name, from a given "scorelist" dictionary as defined above.""" maximum=0 for item in scorelist: value=scorelist[item] if maximumreturn lang
Well, does it work? Quite! I tested it with some Wikipedia text:
SCOREFUNCTION("e Operationen in der Karibik, ohne dass es dabei zu größeren SeeÂschlachten gekommen wäre. In Europa war die erfolglose Belagerung des britischen StützÂpunktes Gibraltar die einzige nennensÂwerte AuseinanderÂsetzung. Der englisch-Âspanische Konflikt endete formell am 9. November 1729 mit dem Abschluss des Vertrages von Sevilla und der WiederÂherstellung des Status quo ante. Die grundsätzlichen Differenzen beider Staaten wurden jedoch nicht beseitigt, was kaum zehn Jahre später zum Ausbruch eines weiteren Krieges führte")
{swedish: 0, portuguese: 0, english: 2, hungarian: 0, finnish: 0, turkish: 0, GERMAN: 5, dutch: 3, french: 1, norwegian: 1, catalan: 0, spanish: 0, russian: 0, danish: 1, italian: 1}
SCOREFUNCTION("Man vet forholdsvis lite om Merkur; bakkebaserte teleskop viser kun en opplyst halvmåne med begrensede detaljer. Mye av informasjonen om planeten ble samlet av Mariner 10 (1974–76) som kartla rundt 45 % av overflaten.")
{swedish: 3, portuguese: 0, english: 0, hungarian: 0, finnish: 1, turkish: 1, german: 0, dutch: 2, french: 1, NORWEGIAN: 4, catalan: 1, spanish: 1, russian: 0, danish: 2, italian: 0}
SCOREFUNCTION("A transit of Venus across the Sun takes place when the planet Venus passes directly between the Sun and Earth, becoming visible against the solar disk. During a transit, Venus can be seen from Earth as a small black disk moving slowly across the face of the Sun")
{swedish: 0, portuguese: 2, ENGLISH: 9, hungarian: 2, finnish: 0, turkish: 0, german: 0, dutch: 1, french: 1, norwegian: 0, catalan: 1, spanish: 1, russian: 0, danish: 0, italian: 1}
But it breaks with non-ascii text (like accents, umlauts and other funny letters,) so it is quite un-useful in these cases. But oh well, for 10 minutes of coding its not that bad, a quick hack.
Since last week I had started to read the django book, I thought this would make for an interesting first project to post online, and you can find it at whatlanguageis.com, with some unicode improvements. Its still in very early beta, working with just a handful of languages and without any kind of text-length checker. Just a proof of concept about my django "skills."
_Beware, in what follows I rant. All figures come from Wikipedia or similar and are expressed with many zeroes and also in written form to make clear what a billion may be._
If you are a regular reader of mostlymaths.net, youll be aware that I dont write a lot about current subjects. In fact, I actively try not to write about whats going on at the moment, one notable exception may be a post I wrote about Mesut Özils stellar debut in the 2010 World Cup. As a curious example, last week we (Laia and me) had a wonderful dinner with Michael and her girlfriend. Hes been a week on holiday here in Barcelona and we found time for a tasty Indonesian dinner. He talked me about a mining strike in Asturias (in the Northern area in Spain,) and I was completely unaware of it. My usual news sources are Reddit, HackerNews and my tweeps. And almost none are Spanish.
But now, I just feel compelled to write about the state of Spain and Catalonia as a byproduct. I am pretty sure out there I could find a decent politician... Given an infinite set of politicians and the axiom of choice. But I dont like doing NP-hard problems by hand anyway, and Ive thought for a long, long while that someday _politician_ will be an insult. And _banker_ could (as well) be another, at least referring to the heads of the bank.
What made me break my standard silence? This article. Its in Spanish, but I can give you a rough summary (or read it via Google Translate)
_A MUNICIPALITY IN CáCERES (CENTER-WEST OF SPAIN) FORMED BY 3 TOWNS HAD 15,000€ PREPARED FOR SUMMER FESTIVITIES... MONEY DESTINED TO BULL FIGHTING. THE MAJOR OF THE DISTRICT HAD A BETTER IDEA: USE THESE 15.000€ TO HIRE (TEMPORARILY) SOME WORKERS, ALLEVIATING THE AREA UNEMPLOYMENT. THE RESULT?_
* _TOWN A: 242-181 for unemployment (40% of citizens voted)_
* _Town B: 47-126 for bull fighting (27% of citizens voted)_
* _Town C: 19-76 for bull fighting (27.6% of citizens voted)_
_The paper interviewed some people. The opinion of a citizen of town A is that "festivities are two days and invoices are due all year long". The opinion of 3 17-year old girls (town unnamed) is that "they are too young to work and they enjoy a lot the acts"._
As the title says, its not only this countrys politicians. Theres a very deep problem here, its not only that a lot of people dont care about what happens to others. In some sense, Im also guilty of this. Its more like people dont realise what is going on. "Fiesta!" They think money grows on trees (or on printers,) in fact, a former minister (from the supposedly left-winged party, now in opposition) said
_We administer public money, and public money is nobodys money._
Yes, this is the idea coming from someone "we" elected. This is the stupid idea running amok in my countrys society: public money comes from nowhere. Somehow I feel it all started 30 years ago with the beginning of our democracy, when to soften Catalonias and Basque Countrys special consideration someone invented the "café para todos" (coffee for all.) What this meant is doing the same in all regions. Barcelona has an airport? Why cant Castelló have an airport, too (and all other regional capitals, even if they have less than 150,000 inhabitants)? High speed train from Madrid to Barcelona? Lets get it going to Cuenca, again.
When the plan started it wasnt as stupid as it is now. Maybe because politicians knew where money was coming from? Who knows. But now the only thing we have is majors and regional presidents building unnecessary infrastructures, after all, who pays the bills? Someone else. Who? Not the Doctor.
And this is the state of affairs. Regions have no money, some less than others. Valencia is completely bankrupt, but they dont care at all. In Catalonia, we are too, and still the central government asks for more cuts. Public employees (who are mostly tenured in all Spain) got a 10% cut (average over all incomes) in Catalonia, but not everywhere else: a general strike of public employees could stop the country. Instead, health and education budgets got a huge cut, while all savings from them go to saving bankrupt banks. The last case, Bankia...
The Bankia case deserves a special post, and there are plenty. For the uninitiated, it was the merge of 7 smaller banks, all of them bankrupt. It should be clear to any lay person that a lot of companies with a lot of debt, joined, result in a big company with a very big debt. Go figure. It was created with money from other banks and government money, trying to clean the worse assets in these 7 magnificent. And who was in charge of Bankia? Rodrigo Rato, former economy minister with the same right-winged party that is now in charge of things in Spain. This same RR stepped down on May 7th, after getting a salary cut from several millions to "only" 600,000 €/year. Its clear I choose (and probably, you too) a bad career: WHEN I DO A LOUSY JOB, I DONT GET MONEY. Clearly the guy did a bad job: he refused a merger with "La Caixa" (the largest Catalan bank) to keep Bankia (mainly a Madrid-controlled bank) from becoming mostly Catalan. This is my "country." Oh, forgot to add this: Bankia had 309,000,000€ of profit in 2011. Turns out, once this was revised, before taxes they had already lost 4,300,000,000 (thats 4.3 thousand million) euro. Im guilty of messing with my calculations and forecasts. But before doing anything with a result, I double (or triple) check. And from my calculations dont depend the money of a lot of people.
Now the state has to give Bankia around 25,000,000,000 (THATS 25 THOUSAND MILLION) EURO to keep it from bailing out. For a lay person, this amount of money is just staggering. Here in Europe we are used to big transfer fees in sports, but they are "just" of the order of 100,000,000 euro (hundred million). So, lets see what 25 thousand million are for Spain
* The GDP (nominal) of Spain is 1,400,000,000,000 (thats 1.4 million millions) euro. So Bankias bailout is equivalent to 1/56th of the total GDP.
* Spanish government revenue is 515,000,000,000 (thats 515 thousand millions) euro. Bankias bailout is ~ 1/20th of the total revenue. BUT
* Spanish government expenses (as of 2010) were already 616,000,000,000 (yes, 616 thousand millions.)
* NASAs budget is 14,250,000,000 (14.25 thousand millions) euro. Almost half of it.
Okay, in short, it is a f**cking lot of money. Money that our country does not have, that Europe cant lend.
What happens when the whole Spanish government bails out? Who knows? Most likely the Euro as a whole breaks... Unless Germany lets the ECB start printing euro bills like crazy.
This is the rant. Not only politicians are stupid, our own countrypeople are stupid. And theres nothing rational people can do about it.
Damn, how Id love to be able to live in Iceland. At least the ratio of shouting-reforming people there is higher.
Further reading (via Google Translate):
* High speed trains, airports and cable cars (theres a Google Translate selector on the right, under "Traductor de la página"
* Bail Out The People (check the Rodrio Rato box)
CortesÃa de Shanidar
Puedes leer la versión inglesa de este post aquÃ: Learn to remember everything: the memory palace method
En este post os voy a enseñar cómo recordar a la perfección una lista. No importa la longitud de la lista: puede ser tu lista de la compra de 10 artÃculos, o una lista con 50, 100 o incluso 1000 cosas. Y en un próximo post, cómo aplicar este método para aprender idiomas. Suena bien, verdad?
La técnica que vamos a aprender se llama el palacio de la memoria, también conocido en inglés como "method of loci" (por la palabra latina locus, que quiere decir lugar,) aunque en ingles lo más habitual es memory palace o mind palace. Este método es una herramienta fantástica de la que disponer!
EL PALACIO DE LA MEMORIA
_El método del palacio de la memoria tiene sus origines en el siglo 5º A.C._, cuando Simonides de Ceos, poeta, atendÃa un (poco afortunado) banquete en Tesalia. Mientras iba a la puerta a atender a un correo que preguntaba por él, el techo del comedor se derrumbó, matando a todos los comensales. No habÃa manera de reconocer los cadáveres (aplastados por un techo...,) las técnicas de CSI no estaban tan avanzadas como en la tele. Pero Simonides se dio cuenta que no tenÃa ningún problema en recordar quien estaba dónde, sin prácticamente ningún esfuerzo.
Piénsalo por un momento: es fácil recordar quien se sienta al lado del anfitrión, dónde estaban tus amigos, quien estaba a su lado. Y asÃ, llenas una cena. A Simonides "se le encendió la bombilla," y gracias a esto está considerado como el inventor del método del palacio de la memoria. Aunque usado ampliamente en la antigüedad, no hay prácticamente registros escritos del método: aparece en la obra anónima _Rhetorica ad Herrenium_ y en la obra de Cicerón _De Oratore_. Pero no es extraño que no haya registro escrito, para ellos serÃa tan normal escribir un libro sobre el palacio de la memoria como para nosotros escribir un libro sobre cómo ponerse unos pantalones. Todo el mundo saber hacerlo.
EL PALACIO DE LA MEMORIA ES UN MéTODO CERCANO A CóMO NUESTRO CEREBRO FUNCIONA. En nuestros dÃas de nómadas cazadores-recolectores necesitábamos saber cómo llegar a algún sitio (lago, llanura) y recordar qué habÃa ahà (agua fresca, fruta, caza). Aprovechando este hecho podemos construir una gran cantidad de métodos de memorización, para listas ordenadas o desordenadas.
Recordar listas puede sonar estúpido, quien quiere memorizar una lista? Pero en el fondo, una lista es un conjunto de conocimientos ordenados! Lo que estudias para un examen de historia es una lista de fechas (ordenada) conectada con hechos (que son sub-listas de la lista principal). Cuando aprendes una nueva receta, es una lista. Un número de teléfono es una lista de números. Un poema es una lista de frases.
TU PRIMER PALACIO DE LA MEMORIA: CONSTRUCCIóN Y LLENADO
Empecemos a crear nuestro primer palacio de la memoria. El hecho que se le llame palacio no tiene que hacernos pensar en Sissy o pelÃculas de Walt Disney, no tiene porqué ser un palacio. De hecho, para empezar, podrÃas usar tu casa, y como ejemplo imaginaremos una casa muy pequeña. Entrando por la puerta llegamos a un pequeño recibidor, que nos lleva a un comedor con 3 puertas. Del comedor podemos llegar a la cocina, al WC y a la habitación, que tiene un balcón. Este es un ejemplo de casa que vamos a utilizar, para usar el método correctamente deberÃamos usar nuestras casas, o otros sitios reales que conozcamos bien.
Ahora memoricemos algo. Una lista de la compra: lechuga, bacon, aros de cebolla, una tarjeta SD y naranjas. He usado una lista corta para hacer el post más corto y para que quepa bien en la casa imaginaria que hemos construido un poco más arriba: prueba con una lista más larga después de esta si no crees que el método funciona!
Para recordar la lista, tenemos que colocar cada elemento en algún sitio de nuestro palacio de la memoria. Esto quiere decir un elemento por habitación o varios elementos por habitación, cada uno en un sitio especial que podamos recordar fácilmente. La manera más simple es poner cada objeto en una habitación diferente. AsÃ, nuestra pequeña casa de 5 habitaciones podrÃa contener una lista de 5, 10 o 15 elementos.
Para colocar un elemento de la lista, tenemos que visualizarlo en la habitación, para asegurar que lo memorizamos, tiene que ser una imagen extremadamente rara. Tiene que dejar una impresión clara, para hacerlo tiene que ser sorprendente, surrealista o sexual, entre otras opciones. Si la imagen es insulsa, recordarla será prácticamente imposible.
Empecemos con la lista. Cuando entramos por la puerta principal, nos saluda la Rana Gustavo, pero un Gustavo especial, hecho de lechuga, como una lechuga parlante. Puedes imaginarlo? Notar el frescor de las hojas de la rana Lechugo? En el comedor hay una estampida de cerdos... perseguidos por Kevin Bacon armado con un tenedor y cara de hambre. Lo bastante raro como para recordarlo? En la cocina, Scarlett Johansson juega con un hoola-hop que es en realidad un aro de cebolla gigante. Entras en la habitación, y te sorprende darte cuenta que la cama es una tarjeta SD gigante, que puedes esconder contra la pared como cuando pones una tarjeta en una cámara. Para acabar, abres el balcón y te encuentras una naranja gigante y brillante haciendo de sol, goteando zumo de naranja sobre el desierto que se abre bajo tu ventana.
Las imágenes debes ponerlas en un sitio que conozcas como la palma de tu mano: tu casa, la casa donde creciste, tu oficina. Esto es muy importante. En este ejemplo he usado una casa imaginaria porque es más fácil, pero en las listas que quieras memorizar, usa sitios conocidos.
Quizá creas que el método no funciona, pero te sorprenderÃa. Escribà la primera parte de este post al empezar la tarde, y ahora, más de 3 horas más tarde al ponerme con la segunda parte aún veo claramente las imágenes. Evidentemente esta es una lista muy corta... Pero no importa: TE SERÃA IGUAL DE FáCIL RECORDAR UNA LISTA 5 VECES MáS LARGA.
ENCONTRAR UN CONJUNTO DE PALACIOS DE LA MEMORIA
Para recordar una gran cantidad de cosas necesitas una gran cantidad de sitios para poner todos esos recuerdos. Necesitarás encontrar tu propio conjunto de palacios de la memoria. La primera vez que me encontré con este problema, pensé en crear sitios imaginarios (como el pequeño piso que hemos usado más arriba,) conectados de alguna manera con pasillos. El problema? Los palacios artificiales se desdibujan con mucha facilidad, y tiendes a olvidarte de ellos. Es mucho, mucho mejor usar sitios habituales, o que al menos puedes comprobar de nuevo en la vida real, como fotos de un libro, niveles de un juego de ordenador o algún edificio al que puedes volver.
Una vez me di cuenta de esto, empecé a pensar en casas y sitios que pudiera usar... Y me di cuenta que tenÃa una gran cantidad. Aún recuerdo (y seguro que tu también!) las casas de compañeros de clase de hace 15 años, hoteles en los que he estado, edificios que he visitado. Estoy seguro que podrás encontrar una gran cantidad de sitios que te sirvan de palacios de la memoria. Para empezar, usa sitios muy habituales, como tu casa o oficina, y a medida que cojas práctica podrás usar otros sitios más antiguos.
Puedes leer más sobre encontrar sitios que usar en este post en inglés: Building Your Memory Palace Collection
PALABRAS FINALES
Tienes que encontrarle el truco al método. Consigue experiencia convirtiendo objetos habituales (como una lechuga) en imágenes que puedas recordar durante mucho tiempo (como Gustavo el cabeza-lechuga). Cogerle el ritmo es solo cuestión de práctica, igual que pasear por tus palacios en tu cabeza. Práctica, práctica, práctica.
Por cierto, PUEDES RECORDAR TODAVÃA LA LISTA DE LA COMPRA?
Te ha parecido interesante este post? Si lo ha sido, cuélgalo en Facebook o Twitter!
_Caveat:_ some of the links appearing in this post are affiliate links to Amazon.com If you buy anything from them, I get a small commission. As always, I only link to stuff I like. If you want to support (ever so slightly) this blog, buy something. If you dont want, dont do it ;)
Lately Ive been watching an interesting TV series. Sherlock, the modern version of Conan Doyles stories and novels. It is written by Steven Moffat, one of the writers of the new incarnations of Doctor Who, another series I enjoy a lot. Two seasons have come and go (they only have 3 chapters each,) so far both excellent. But Im not writing this post to (just) praise a TV series, after all Im not a TV fan whatsoever. What I want to highlight is the appearance of the "mind palace" (as Sherlock named it) in some chapters, another way to put the more widely used memory palace. I have already written a post about how to use the memory palace technique, working as quite an introduction to the subject. But I want to retake it again, since some popular occurrences of the memory palace are pretty... odd.
Setting aside classical titles on memory or memory techniques, and the (somewhat) known book The Memory Palace of Matteo Ricci, the first best-seller book I know giving some hints about the method is Harris Hannibal, the prequel to Silence of the Lamb. In it, Hannibal Lecter explains how he stores all his memories and knowledge in an intricate memory palace. Anything from poems to maps. In _Sherlock_, Sherlock Holmes is also depicted as using this method to remember a London map, before chasing a cab using his knowledge of one-way routes and construction sites. In another chapter, he is also shown deep in search of data in his mental library. But is it really like this how it works?
First, for newcomers who dont want to read my lengthy article about the memory palace method, lets make a thought experiment to realise how powerful the method is. Close your eyes. Can you picture your parents (or grandparents) house? Mentally walk from room to room? I bet you can, even if your parents have moved since then. Can you walk mentally the route from your parents home to your school, or high school? Im sure you can do it without any problem. Your brain is hardwired for this: finding routes. After all, a hunter-gatherer unable to remember where the plants or the water lay was doomed to die.
The memory palace method aims to take advantage of this ability, paired with our visual memory and linking memory to remember... lists. In fact, you dont need to remember lists, but the method is best suited to listed knowledge, since you can remember it ordered. The simplest example would be a shopping list. If you want to go shopping and need milk, onions and some tasty dressing (making some onion rings, maybe?) you can place each item in... your grandparents home. Or along the route to your high school. Problem is, done plainly forgetting its too easy. You are likely to walk over the 3 onions you placed in the middle of your parents living room. To remember, you have to make everything bigger, noisier and bizarre. Cleopatra (the Egyptian pharaoh of old) bathing in milk in the bathroom (in case you didnt know, Cleopatra is said to bathe in donkey milk). A giant onion-man eating sliced humans in the living room. A bride (well dressed with her wedding dress) covered in ketchup. Far easier to remember, dont you think?
On the other hand, the fictional cases of memory palaces showin in _Hannibal_ and _Sherlock_ are far more abstract. How are you supposed to remember poems or a map? Well, all you need is a code. To memorise a poem you need to remember its verse and rhythm. For Keats Ode on a Grecian Urn, youd start (again) with a bride, dressed in white with a finger to his lips, ordering silence. Then a boy appears out of nowhere, carrying a huge golden pocket clock in his neck, also gesturing for silence. And so on and so forth. One (or two) images for each verse are almost all you need. Pair it with some repetitions of the poem to get the hang of the rythhm and youll never be able to forget it.
And how are you supposed to remember a full map? Well, this one is trickier and I still dont know how to remember a plain map. Remembering directions is somehow easy: just store a direction in each room of your palace: turn left, go straight and right on the 3rd is just "1 left" "straight 3 right", which are easier to encode in rooms in a palace. For a map, I dont know and I dont think Sherlock or Hannibal could help us.
Of course if you have a good enough visual memory (or even eidetic memory like Sheldon Cooper in The Big Bang Theory) you could remember the full map. But this is just being born with the right genes, and Id rather know how to do it myself.
If you enjoyed this post, please share it with your friends.
Just in case you dont know, vi is an advanced text editor, drting back from the same era as emacs was developed (emacs started slightly earlier). Sort of the Jekyll to emacs Dr Hyde. Emacs users despise vi users, and vi users mock emacs users. This is what the editor wars are all about: "Eight Megabytes And Constantly Swapping" versus "vi has two modes: writing and beeping". If you have been long enough in this blog, you know Im in the emacs side, but you also know Im curious enough to delve into the other side.
My first contact with vi was in our first programming class at the University, _Informatica_ (Computer Science). Vi was the editor of choice for the course, so much that you did not have any other option given. Along with a Linux cheatsheet and a short C manual came a vi cheatsheet. A lot of people came to hate vi, a lot of people came to love it. I was more in the hating side, but it wasnt that bad. After a while I just started using a "normal" Linux editor (it was Kate, maybe?) and kept using it (or UltraEdit when I was in Windows) until the end of my degree.
When I finished my degree I had to be a teacher of _Calcul Numeric_ (Numerical Analysis), the third (and last) programming class in the degree. At home I was still using Windows, because the wifi card in my notebook had no Linux drivers and in my office we were using Ubuntu. I needed a cross-platform editor that was quick, versatile and could last for a lifetime. I tried vi (again) and emacs, emacs won the war for two reasons. First was the fact that it was extensible programming in Lisp, and I had already a decent knowledge of Lisp. The second was AucTeX and its "preview" option for LaTeX editing. Nothing can beat pressing C-p C-p C-d and seeing your formulae come to life in your editor. I became an emacs advocate for this reason, then after 5 or 6 years Ive found many more reasons why this is THE editor. Or, the operating system, if you prefer.
But advocacy is nice, but knowing all the players in the field is better. My department pals used vi (at least most of them), and it was impossible to convince them to switch. I decided learning vi (or vim, vi iMproved) was something I had to do some day, to learn what was there. After all, if I liked it so much I could set viper-mode in emacs and use vi keys in emacs. The best of two worlds, if that world is so nice.
Then the guys at Applidium released a Vim port for iOS devices. Whoah! Even if Im an emacs guy to the bone, a modal editor is way better than anything else available in the app store, at least for raw editing power. And I say with knowledge, because Ive tried most writing apps for iOS devices (see this post).
As an additional reason, Matt Might posted a New Years post detailing several resolutions for programmers. Among them was breaking your comfort zone to keep your mind sharp. The first example? Switching to vi from emacs or vice-versa. He used to be an emacs user before getting to use Vi. I dont think I could go that far, but these were enough reasons: advocacy (from me, not against me), iPad+bluetooth keyboard and breaking my comfort zone.
HOW IS THE EXPERIENCE SO FAR?
Well, Im writing this post in my sofa, with my iPad
and my bluetooth keyboard sitting in my lap. Im still getting used to having to exit editing mode to move around: Im very used to pressing C-a or C-e to go to the beginning of end of line. An additional problem is that even with the bluetooth keyboard, Esc cant be mapped to the Esc event, and I needed something to do it. I have jk (pressed together) for it. It was a suggestiond I saw in Hacker News comment thread for the release of the app, and its quite handy. Not so much with the on-screen keyboard, but for now its okay. To remap this, press backslash (the current mapping for Esc) and then :imap jk
and my bluetooth keyboard sitting in my lap. Im still getting used to having to exit editing mode to move around: Im very used to pressing C-a or C-e to go to the beginning of end of line. An additional problem is that even with the bluetooth keyboard, Esc cant be mapped to the Esc event, and I needed something to do it. I have jk (pressed together) for it. It was a suggestiond I saw in Hacker News comment thread for the release of the app, and its quite handy. Not so much with the on-screen keyboard, but for now its okay. To remap this, press backslash (the current mapping for Esc) and then :imap jk I found out also how to change the font size, at least for now this works (maybe there will be more fonts available in the future), type :set guifont=Courier:h24 (for 24 pixels)
To round everything, I just installed the famous solarized color theme, dark in vim for iPad. Until now I had just thought it was an overhyped color scheme: I had tried in my emacs and didnt enjoy it that much. But in my iPad it shines with a distinct colour, its so much easier on my eyes that it hurts to get out and write an email! To install it in your vim for iOS, first download solarized.vim from the git repository, then plug your iDevice and use iTunes File Sharing to copy this file to the vim app. Then open vim, :e solarized.vim and :call mkdir(.vim/colors/) (to create the directory needed for it) :w .vim/colors/solarized.vim This is just because I didnt seem to get iTunes File Sharing to work with hidden files (a file starting with a dot is hidden). Then you have to add the following to your .vimrc:
it shines with a distinct colour, its so much easier on my eyes that it hurts to get out and write an email! To install it in your vim for iOS, first download solarized.vim from the git repository, then plug your iDevice and use iTunes File Sharing to copy this file to the vim app. Then open vim, :e solarized.vim and :call mkdir(.vim/colors/) (to create the directory needed for it) :w .vim/colors/solarized.vim This is just because I didnt seem to get iTunes File Sharing to work with hidden files (a file starting with a dot is hidden). Then you have to add the following to your .vimrc:syntax enable set background=dark colorscheme solarized
I recommend using this app for any emacs lover out there with an iPad. Of course, it is not emacs, but vim is a pretty awesome text editor, and its always handy to learn to use it.
To have a useful list of vim commands at hand I installed the Vimmy app, a universal app with the most common vi commands. I can switch to it via the multitasking gestures in my iPad, or use it in my iPod Touch while Im using it in the iPad.
CONCLUSION
I dont think Ill ever switch from emacs to vim any time in the future, as I said AucTeX is definitely too good. Of course since then Ive found many more things I love: having a REPL for Python, Clojure or Lisp inside my editor. Theres even a REPL for PostScript! Also local remote editing with tramp... There are too many things I use on a daily basis and I cant barely remember, they are so entrenched in my .emacs file I cant even realise what they are. But for my iPad it is an awesome addition to write text on the go.
Again, if you have never used vim before and are interested, give it a try. Of course the first time you open it it will be... a jump into the unknown. Tap the screen, press :e filename and start editing happily. Esc (well, jk or backslash, as the initial mapping) :w to save. And you can even :q to exit to springboard!
Taken from Flickr The links to books in this post are affiliate links! Beware :)
In case you have not realised it yet, Im a pretty prolific reader. Online reading (and having an iPad) have slowed down the number of books I read in a given year, and I dont go to the lengths of my girlfriend (who is about to reach her goal of reading 102 books in this year,) Im nevertheless a frequent reader.
This year Ive read several good books that Id like to share with you, after all, if you are reading this probably our tastes overlap. And then these books will be perfect for you (or for some geek in your life). The books Ive selected range from programming, sports and non-fiction. Oddly enough, I dont remember any fiction book Ive read this year (not counting some book by Raymond Chandler, which I can recommend each year). Lets go!
BORN TO RUN _A HIDDEN TRIBE, SUPERATHLETES, AND THE GREATEST RACE THE WORLD HAS NEVER SEEN_: I reviewed this book in a previous post here. I found it very good, and urged me to get running. My running frequency is (to put it softly) awful, but Im trying to improve. If you want to start running or recover some lost motivation, give it a shot, its a very good read. You can read my review of Born to Run.
VISUALIZE THIS _THE FLOWINGDATA GUIDE TO DESIGN, VISUALIZATION, AND STATISTICS_ I wanted a book to help me graphing things in my future work as a freelancer, and after looking around a little I found this *great* book by Nathan Yau, the blogger behind Flowing Data. Its an awesome book, which will help you start kicking around the graphs and tables.
CONFESSIONS OF AN ADVERTISING MAN: An almost autobiographical book by famed ad-man David Ogilvy, if you are anything into copywriting, advertising or marketing, you have to read this book. Not only will it help sharpen your ideas, but it is also a very good and entertaining read.
MOONWALKING WITH EINSTEIN _THE ART AND SCIENCE OF REMEMBERING EVERYTHING_: A New York Times best-seller by Joshua Foer. After reading his interview-article there, I decided to buy it (pre-ordered it!), and it was very worth the money. An entertaining read of how motivation can get you a long way, and how some memory techniques work. I have not written a review of it, but you can read my own explanation of the memory palace technique.
EVEN A GEEK CAN SPEAK _LOW-TECH PRESENTATION SKILLS FOR HIGH-TECH PEOPLE_: The best book for anyone in need to give a presentation. Clear, concise and to the point, this should be a forced reading for all technically minded people. You can read my review of Even a geek can speak.
MINING THE SOCIAL WEB _ANALYZING DATA FROM FACEBOOK, TWITTER, LINKEDIN, AND OTHER SOCIAL MEDIA SITES_: If I had to try the book I enjoyed the most reading this year, it is this book. Seriously. Even awesome comes short of what I think about this book. I plan on reviewing it, but Ive got not enough time yet... With it youll learn how to mine twitter, Facebook, LinkedIn and whatnot. And then process all this data. Easily. Yes!
THE ADWEEK COPYWRITING HANDBOOK _THE ULTIMATE GUIDE TO WRITING POWERFUL ADVERTISING AND MARKETING COPY FROM ONE OF AMERICAS TOP COPYWRITERS_: Your deep entry into copywriting, from a master copywriter. Youll learn how to shape your online copy, from headline to action. Its very well written in an understandable fashion. Keep in mind that if you buy the Kindle edition, the provided checklists are not that useful.
THINKING FORTH: Ive had this book for a long, long time and I finally got to read it. I used a simple method for getting me to read it: I left it in the bathroom. When I had to spend some time there (for example, shaving with an electric machine!) I read a few pages. And in no time, I had finished. You dont need to have a computer close to read it, but a little knowledge of Forth is quite useful. It is a clean book, introducing some concepts like orthogonality and testing without even trying.
POWER SLEEP _THE REVOLUTIONARY PROGRAM THAT PREPARES YOUR MIND FOR PEAK PERFORMANCE_: A best-seller book focused on mental abilities and how sleep affects it. Packed with research results and advice, it was a very interesting book, although a little too long. After all, the best suggestion is to go to sleep each day at the same time and keep adding 15 minutes (weekly) to your sleep until you stop feeling sleepy. Now that you have the spoiler, buy it.
CRUSH IT! _WHY NOW IS THE TIME TO CASH IN ON YOUR PASSION_: One of these self-help books that will get you pumping ideas and rocking you off the sofa. According to Gary, now it is time to crush it. And probably it is. If you are a little low on motivation, read this book. Dont expect it to teach you how to do things, just read it as experiences to think about.
WHAT I TALK ABOUT WHEN I TALK ABOUT RUNNING: Another running book, this time an autobiographical piece by one of my favourite writers, Haruki Murakami. I was expecting a little better from him... But of course, not every book can be Hard-boiled wonderland and the end of the world. An interesting read about Murakamis running experiences, and how he got to run a 50 miler and the "original" marathon in Greece (under a scorching sun).
This month Ive started several books that Im sure would have make this list, and will probably be delayed until next year:
STEVE JOBS: I will remember how and when I found out Steve died, and will still be inspired by his commencement speech for a long while. Ive gone through almost one fifth of the book (a very thick one,) and it is one of the best biographies Ive read (the other is John Nashs). Brilliantly written, it reads like a novel with a fast pace.
1Q84: the recent best-seller by Murakami was the gift from my girlfriend for this years St. George day, and Ive only been able to read a few pages. From the raving reviews, I assume Ill like it (I was deceived by Kafka in the shore).
THE MEMORY PALACE OF MATTEO RICCI: A classic book for memory techniques lovers, Ive only got to read the first 20 pages or so. So far, very interesting and well edited.
GUNS, GERMS AND STEEL _THE FATES OF HUMAN SOCIETIES_: A book recommended by my thesis advisor, dealing with why Europe invaded America and why it was not the other way around. Of course, all is theoretical, but it is amazing nevertheless.
If you find some broken link, please let me know!
.png "All hail Steve Jobs")
Some of the links in this post are affiliate links. As usual, I only recommend what Ive used and like
Inspired by a post by Mark OConnor from Yield Thought (my frequent readers will have already read something from him from my link collections), I have been working remotely for a week. His set-up is an iPad 2, Apple wireless keyboard, the iSSH app and an account in Linode. My setup is similar, but I use an iPad 1 and 6sync for the VPS.
WHY 6SYNC?
A matter of luck. Around 2 weeks ago, I set up my "name domain" at rberenguel.com (currently down, I messed the Apache configuration a few days ago), and I decided I had played with blogger for long enough. Decided to look up an VPS to host that site and whatever I needed to either host or cloud process in the future. Tweeted asking for options, and in the meantime, Sacha Chua explained how she switched to Linode. Linode was going to win...
Until the guys at 6sync (Mario and Rav) sent me a tweet, urging me to check 6sync before choosing. I checked the page, the prices, and was close to sold out. Exchanged a few tweets, read a few reviews, and I was convinced. Reasons? Many.
First, they offer a more basic and cheaper service. Whereas Linode offers 512 MB of RAM, 20GB storage and 200GB transfer for 19.95$/month as the basic plan, 6sync offers a nano plan with 256MB of RAM, 12GB storage and 150GB transfer for 15$. This plan is currently ideal, since my bandwith and RAM usage for the server side is close to nil, and I dont expect to use that much of bandwith in daily doses. They also offer a Mini plan with 2 CPUs, 512 MB or RAM, 20GB storage and 400GB transfer for just 20$/month, beating the Linode offer on its own ground.
Second, they have a very nice website. As a hobbyist page creator, this is important. I like how they have set-up their site, and the pictures they choose to go by. If they are not the designers, the designers should receive this praise.
Third, they are a funny bunch. I exchanged several tweets with them, and they were very supportive. And as soon as I had my server up and running and ran into problems (or more precisely as it turned out, I was not patient enough for the DNS switch) they were extremely quick to help through their in-page customer chat service.
Fourth, they have a lot of server setups ready. Do you want Debian? Check. Ubuntu? Check. Arch...? Yes! I can use me dear pacman package manager and install whatever I want, in the latest version. Cool.
All in all, very happy for hosting. And then, Mark published his post.
THE REMOTE AND LOCAL SETUPS
At first, instead of filling my 6sync server full of software, I tried a simpler setup: using my office computer in the department. I still have an account for using that computer, and I have already everything setup for working remotely. I only needed a decent ssh client for iPad.
Back in the iPod Touch days, I used the awesome (truly awesome) TouchTerm app, which is (was) perfect for the iPod Touch. Offering a wide array of gestures to control it, it is a marvel of usability. But it has no iPad version. For the occasional SSH session I was using the free zatelnet app. But this also was not a good deal.
When Mark published his post, I decided to buy iSSH, and even if the price tag is high for an app, I dont regret it. The freedom you get when only carrying an iPad and a very slim bluetooth keyboard is _unmatched_. The app is universal, so you have two apps for the price of one. Check the server stats from your iPhone!
Mark is a vim guy... I guess thats okay for a lot of people, but not for me. I could go the vim way (I have been trying to get a basic proficiency of vim in the past), but Im an emacs guy. Up to the last moment. And I had to cross fingers for iSSH and emacs to play along nicely...
Well, they mostly do (at least with iOS5). The control key works, and so does the option key (as meta). The real meta key (command) does not work as expected, but esc also works as meta, in case of need. The only problem Ive found so far is the fact that control-f and control-b are hard-bound (in the iOS level) to advance/go back a word. Thus control-x control-f does NOT open a document. But if you press shift, emacs does not care about control-X control-F, and the iOS bound setting is overriden.
Theres a more important problem with these iOS level shortcuts, control-a goes to the beginning of the line... but leaves a nasty [] in the buffer. I have written an abbrev-expansion and re-bound control-a to get rid of it, but it does not work as well as I want yet. Luckily emacs is flexible enough to allow me to solve this in more than one way. Currently almost done anyway.
Installing all the stuff is a breeze. Of course, I had already installed the LAMP setup, although I only have configured Apache so far, but adding the rest of things I use on a daily basis is straightforward:
pacman -S emacs-nox texlive-core
And now... Dropbox or not dropbox? Dropbox from the command line has a slight problem: you need a browser to accept the linking between your new computer and the remote system. And I didnt want to add the complexity of X to the game (even if iSSH has a very cool X server...). So I left dropbox out of the equation. I can always push/pull the files every once and then without problems.
So, I have everything that I need. Now, how does it work in practice?
THE DAILY GRIND
As you may already know, my _daily grind_ is split between my part-time freelance job as _social media manager_ (among many other things covering _get more pageviews and make more money from the sites_, mixed with data analysis and copywriting) and finishing my thesis.
Working in my iPad for the freelancing job was already possible. I could access almost all systems from just a browser. Reading and answering mails, the same, although going from the email app to the CRM is slow. So far, the only problems Ive found are spreadsheets (solved since I got Numbers) and e-Junkie. e-Junkie has the nice feature of having a flash-based login box. Clever, isnt it? Well, Ill leave the e-Junkie management for my MacBook moments.
For the thesis part, it was harder. There is an awesome LaTeX editor for iPad, TeX Touch. I cant really stress how good it is. Using it with the on-screen keyboard is as close as you can get to be proficient. The drawback is that once youre used to emacs, if you have a solid keyboard, nothing can beat emacs except telepathy.
This is where the iSSH stuff enters into play. Fire emacs, edit all your TeX in console mode (I can somehow live without AucTeXs preview, although it is slightly hard), compile the pdf and move it to the shared folder (all done via an automated script within AucTeX for compiling). Switch to Safari (or GoodReader) and open the document. Check everything is in place and keep going.
The standard TeX based lather-rinse-repeat is straightforward from within an iPad.
Anyway, Im not using only the iPad day to day. Only those days when I know I dont have that much work to do, requiring heavy task-switching or firing tons and tons of emails with attachments. For slow (or semi-slow) days, it is awesome.
WHAT BENEFITS ARE IN THIS?
Well, Im free of carrying a heavy MacBook. Until I get a nice MacBook Air, my current computer is a slightly too heavy MacBook from 2008. It weights quite a lot, and I need to add the charger (even if it is very light, it weights). When I carry the MacBook, I also carry the protecting case (more weight) and a backpack (weight) with some folders with the stuff Im correcting for the thesis (I could remove this). And I also carry my iPad.
These days I get by with my purse-bag, which is perfectly sized for the iPad+keyboard. I carry the pages I need to check for the thesis, and no more. So Ive gone from 5-odd (or 6-odd) kilograms to slightly more than 1.
I also dont need to worry about plugs or battery, the iPad battery lasts for a days worth of working.
A few weeks ago I found out the awesome time tracking setup that Sacha Chua has in her homepage. Although Im a regular at her blog and I follow her RSS feed, I had not seen it "live" yet. My inner geek yelled "gimme gimme" and I asked her what mobile software is she using for time tracking. Sadly (well, _sadly_), it is Android only app. Oh well. I dont want another device. So how can I track my time _always_, when the only thing I have always with me is either an iPad Touch or an iPad?
There is a problem with the App Store. Usually, as the advertisement says _theres an app for that_. The problem is that Im very picky. I want an app that does exactly this and that, but I dont care about this other fluff or this nice buzzword.
I asked myself what I wanted for a time tracking app. In the end, I want just one thing: timestamps and subjects. Nothing else. Something that may look like:
20111105 2103 2125 blog
Simple, isnt it?
Date | start time | end time| subject
I dont need graphing, or aggregating, or Dropbox syncing, or any other such thing in my iPod Touch. I dont even need length, because this can be recomputed later. I can do all these things far more effectively with Python and R and/or Google Fusion tables in my computer, for example. I could even do it with Python and Gnuplot, or if I was truly desperate, awk and Gnuplot.
Now, the funny thing is that even if I dont have a developer account with Apple, or iOS development knowledge (yet), theres a quick way to create a program that does exactly this for my iDevices (in fact, for almost any device with a browser).
HTML5+Javascript (using localStorage and cache mode). Simple, isnt it? Now I have a small app to track my time in my iPod Touch. Just a few lines of Javascript and a few buttons:
* A button to timestamp, press it again to end the timestamp
* A button to dump all the data from localStorage and show on screen
* A button to clear the data from localStorage
* A button to mail the data
Nothing else. Nothing more.
Each time the timestamp button is pressed, a nicely formatted string gets saved into localStorage, making sure everything works even when I lose the local cache copy due to refreshes. The dump data button is to know what I have already logged, before mailing. And of course, even if 5MB is quite a lot, I dont want to fill it so I need to clean up when I send.
For now, I need to refactor the code before posting it. I finished it in two batches of 20 minutes, and in the second I gave up on refactoring, and clean a little the functionality. But its simple and it works, I have already tracked more than a week of data. Now I only need to find the time to write the analysing code. So far I have a Python function to generate the spans from the time stamps, but Im still not sure how I want to analyse everything. If I want to generate aggregate data from Python (Im more fluent) or in R (more versatile, and I will do the graphing here).
I dont usually post on two consecutive days... But this post is special: today is the 20TH ANNIVERSARY OF ANOTHER WORLD, one of the best games I remember from my childhood. And to commemorate, they have released an iPad/iPhone port. Which is awesome!
5 Years ago, for the 15th Anniversary, another special edition was created for Windows with high resolution graphics, and this is the base version for the iPad port. It looks awesome, and you can switch from the "old" low resolution graphics to the "new" high resolution while in-game, to get a taste of what games where in 1991.
I have only played a little, mostly because I should be working. You have two choices for the controls: touch and pad. The touch controls look very well thought, but after many (many!) deaths in the original (and 15th Anniversary) editions of Another World, I prefer pad.
Also, to appeal more players, they have created 3 difficulty levels: _Normal_, _Hard_ and _Hardcore_. I have not checked, but I bet the original was only Hardcore. Will need to play more to check, which I will undoubtedly do. By the way, all the links to App Store in this post are affiliate links... But you bet this game deserves it.
This was posted to my mail newsletter a few months ago. Subscribe now if you want to receive this kind of content to your inbox. I wont spam you, promise.
When do you have your best ideas? If you are anything like me, I have my best ideas when I cant act upon them. While Im falling asleep or while Im taking a shower. And a lot of them get lost forever, I always think Ill remember it next morning, something that never happens. But I found a nice way to solve this, with a solution very similar to the memory palace technique for remembering lists. USE A MENTAL DRAWER.
I have to confess that the idea is not completely mine... I read it a long time ago in the preface to Four Past Midnight
, by Stephen King. He uses it for drafting his book ideas. King puts ideas in the drawer and lets them grow. We wont be using the drawer like him, but the idea is pretty similar.
, by Stephen King. He uses it for drafting his book ideas. King puts ideas in the drawer and lets them grow. We wont be using the drawer like him, but the idea is pretty similar. You need to imagine your mental drawer (design it as you wish! modern Swedish style or 18th century wood, as you please), as vividly as possible. Feel the smell of wood (or lacquer), the sound as it opens and closes, the feeling of its surface. Keep it in mind for a few days, mentally opening and closing it as soon as you wake up or get out of the shower.
After getting used to it, you are ready to harness its power. Imagine you have an idea before falling asleep, for example that youd like to interview Steven Pressfield (like I did, read here). You want to keep this idea, as it sounds interesting and worthwhile but dont want to turn on the lights and write it down: you are almost asleep, turning on the lights will mean 30 minutes before falling asleep again. Now you have a solution: mentally open your drawer and put (imagining it as vividly as possible) something related to Steven Pressfield. For it to work more effectively, the image has to be bizarre. You could imagine Steven Pressfield squirming inside the drawer like a contorsionist, or a bunch of Spartan soldiers getting out of it (Pressfield wrote Gates of Fire, about the Battle of Thermopilæ).
As soon as you wake up, you check your mental drawer, notice Steven inside it and write down the idea in your notebook. EASY AND EFFECTIVE. Of course, for it to work you need to make checking your drawer a daily habit, but this is another whole story Ill write about any other time. Stay tuned, and share with your friends if you liked this post.
_This technique will be a small part of an upcoming ebook on memory techniques Im currently writing. If you are interested in being a "beta reader" or even being an affiliate/reviewer, just drop me an email. Dont be shy!_
Related posts in mostlymaths.net:
* Learn to remember everything: The memory palace technique
* Remembering facts: mental associative chains
Image courtesy of manatari @ Flickr
As of late, Ive been playing a lot with data analysis and visualization tools. Recently Ive read two interesting books (Statistical Analysis with R
and Visualize This: The FlowingData Guide to Design, Visualization, and Statistics
) and Im on my way to another two to refresh my statistics knowledge.
But this post is only mildly related to these books, since it started way before: the day I read about Gephi. Gephi is an open source graph visualization tool, to work with huge (or at least very big) datasets and graphs. Ive seen it used to graph the friendship network from Facebook, or to graph tweet-retweets from Twitter (these two can be found in the Learn section from the Gephi website). In this post Ill guide you step by step to do a keyword-landing page directed graph, using data from your Google Analytics account.
A directed graph is a graph with arrows. You have a set of sources (in this case keywords) and targets (in this case landing pages), and aggregate sources by name. Plot this, and you have a keyword-landing page graph like this one (data for the last month in mostlymaths.net restricted to the first 500 keywords).
and Visualize This: The FlowingData Guide to Design, Visualization, and Statistics) and Im on my way to another two to refresh my statistics knowledge.
But this post is only mildly related to these books, since it started way before: the day I read about Gephi. Gephi is an open source graph visualization tool, to work with huge (or at least very big) datasets and graphs. Ive seen it used to graph the friendship network from Facebook, or to graph tweet-retweets from Twitter (these two can be found in the Learn section from the Gephi website). In this post Ill guide you step by step to do a keyword-landing page directed graph, using data from your Google Analytics account.
A directed graph is a graph with arrows. You have a set of sources (in this case keywords) and targets (in this case landing pages), and aggregate sources by name. Plot this, and you have a keyword-landing page graph like this one (data for the last month in mostlymaths.net restricted to the first 500 keywords).
Keywords (arrow) Landing pages
WHAT INTERESTING THINGS YOU CAN READ FROM SUCH A GRAPH
* Clustering: In this particular instance, I can find the "clusters" of my blog. You can see 3 big aggregations of keywords and landing pages (left-middle,up-center and right-middle). Each one of these marks pages and groups of pages with several landing keywords. The groupings mark like the "big themes" in my blog, at least from search engine traffic (these are memory techniques, 9 best programming books Ive read and seed germination respectively). Oddly enough, Ive had one visit landing in seed germination from the keyword "gnus mail", thus the big seed germination cluster includes emacs.
* Big keywords: In gephi you can scale nodes depending on other table variables. For example, I can scale the landing page dots depending on the number of keywords, or the number of visits. In particular, I can show which keywords lead to more of my pages, by using the outDegree (number of arrows escaping) of a node.
* Big pages: Alternatively, I can scale nodes with the inDegree (number of arrows entering) to learn which pages have the most different keywords. I can also label the arrows with the number of visits (and scale them with this metric).
* Huge pages: with a little more work, I could scale with total visits to see which are my biggest pages together with the landing keywords.
STEP BY STEP GUIDE
Ill assume that you can manage to download and install gephi on your own, and have some knowledge of Google Analytics. Of course, the first you need is to select the keyword and landing page fields in your Google Analytics page:
Then select Show Rows and select 500:
And get a nice CSV file with the export tab on top:
Remove all this, below Table you should have the useful data,
Now I dont know if this is because I select CSV and not CSV for Excel, but you have to clean the file: there are a lot of table identifiers to remove before the real data table. You also need to remove a line at the end of the file. You can do all this editing with any plain text editor. I used emacs.
Remove the last line with dashes. Now the file is almost ready to be imported... You need to rename the Visits column to something else, for some odd reason Google Analytics copies the same column twice and Gephi needs unique column names. Ready to import! In Gephi, go to the Data Table window from the Window menu, and click Import Spreadsheet:
And now you should select edges table and comma-separated:
Oops! More tweaking of the CSV file is needed: Change the first two columns to Source and Target (and dont forget to mark comma as I did, too) and re-import the file:
After clicking next, Gephi will ask you for the types of the columns. Since Im only interested in visits, pages/visit and average time on site, these are the only columns I want to be floats. You need them to be float to scale the nodes according to these parameters:
Next, well see a blob of points with no labels. Choose the Force Atlas algorithm in the Layout window, run for a while and then click stop:
The result from the Force Atlas algorithm to distribute points will look nicer, spreading aggregates of arrows all over the place. Now we need to get the labels. To do so, you have to click on a somewhat hidden button in the bottom-right part of the Graph window, and select the correct names for the tags. In this picture Ive also marked the label and edge scaling according to parameters:
Once you have labels, you need to play with the layout to make them more visible. Run the algorithm Adjust Labels for a while, then Force Atlas and repeat until you are happy with the results:
You can also set labels for the edges, if you are interested in seeing visits, for example. But I found it messy and unchecked the option. Finally, to scale the nodes, use the Ranking window:
The arrows mark the different ways to adjust: you can change colour, size, text size or text colour. Feel free to play with all these settings!
You can get a nice PNG file of you graph by pressing the camera-like icon in the Graph window (clicking the down-arrow will list the properties), or you can export a PDF or SVG with some more work from the Preview window. Both result in very nice images, but a SVG or PDF can be tweaked extensively using Illustrator or Inkscape. But it also needs more work from the Gephi side, so I wont enter into the details in this tutorial. Well, just a little! Be sure to check "show" before exporting, or youll see a blank page:
And with just these steps you can create quite nice (and useful) graphs like this one:
And here you can find a closer zoom:
I hope you enjoyed this tutorial, if you did it would be awesome if you share it. Ill try to do a few more Gephi tutorials as I find more interesting (for me) uses of Gephi. I have also a few post /tutorial ideas about visualizing Facebook Insights data for pages and using Google Fusion Tables. Stay tuned!
From kevygee@Flicker
Id like to apologise for my lack of writing as of late. See? I just broke one of those absurd blogging conventions. Sincerely, I dont give a d––n about doing that. Please, keep reading.
This is one simple example. Blogging gurus say that you should not apologise for not writing, since that would seem pointless to your readers and youd appear like a less capable guru. I say _so what?_. Im not into writing here to make tons of cash (they are very welcome if they come anyway), and I have a lot more things to do beside blogging. Im here to write for my readers. As of now and according to Feedburner, 620 something. ID LIKE TO THANK EVERY ONE OF YOU FOR COMING BACK. I try to thank you by writing interesting stuff.
Its been quite a while since my last post (which was a book review, not even a "real" post). And why is that? There is a mix of reasons and motivations going here.
Although I had my decent share of ideas, none were compelling enough to get me into writing mode. If Im not pulled by the idea, the post turns out dry or boring. I dont want to write boring posts. My ultimate goal is to be able to write a post about some theme and find that readers uninterested in it can read it whole and enjoy the experience. I follow some bloggers like these, I love their ability to keep me entertained.
This is closely related to another stupid convention in blogging. That you should only write about one theme, only one niche. If youve been here long enough youll know that I write about anything I find interesting. Programming, time management, whatever I find interesting. I could not write about just one thing in _my_ blog. Im like that, I enjoy a hundred different things. Just like everybody else.
I also have an unwritten (at least until this post) deal with my readers. I dont want to write "empty" posts. I dont want to write 700 words of fluff, akin to political speeches. I want to get information out there, share what Ive learnt, ask what I dont know. Let the politicians (and quite a great deal of bloggers, too) talk or write for hours/pages without saying anything.
Im not here for the money (as I have already said), but for my readers. You can see ads around the page. Currently they are:
* A spot below the menu, cycling through Bookdepository (50% of the time) and two different AdSense colour schemes (25% and 25%),
* An Amazon my Favourites widget in the sidebar.
And thats it: you see, I wont make any decent money. A few months ago I dumped AdSense because I wanted a more ad-free experience for my readers, leaving only the bookstore ads (I love books and love to share books). AdSense is here again because I have started an A/B test with AdSense, just for fun. I want to see which colour layout is more effective, to do so Im running this ad splitting scheme for the different colours and in a few weeks Ill know which one is better. Im curious, its not that AdSense makes me a lot of money (2 or 3 euro each month, at most and with luck).
Another reason Ive been less eager to write lately is that Im busier, and it gets harder to fit writing quality and informational articles (DISCLAIMER: this is what I _try_, I dont promise all my articles are up to my tastes) into my schedule. Also, ideas are like __________. You can come up with almost anything and find some analgoy to comply. In this case, I feel like IDEAS ARE LIKE MUSHROOMS. Why is that? If weather is very hot and dry, there are no mushrooms. But come a big storm and they will pop all over a place. This happened yesterday, for unknown reasons.
All ideas I had lately were now seen in a new light, and I started filling a notebook in Penultimate for iPad (an affiliate link, this app is quite neat for this quick sketching of ideas or taking meeting notes) with post ideas and execution (screenshots, tables, screencasts) for some of them, Id get them written soon. Also, I decided to write this post, because the ideas in it were around my mind for quite some time and needed some pouring.
Ill never promise I will write more frequently. My only promise is that I try to do my best to write interesting stuff. I owe this to my 620 something readers and also to everyone else that stops by this blog. Thank you all, and keep coming.
Copyright © 2009 COD, All rights reserved.