REST in Place has never been more than a proof-of-concept for me. It worked, pretty well even, so I never bothered much to develop it further, especially since integrating JavaScript code with Rails apps was a bit of a pain. I did only the barest, necessary maintenance and never actually realized that there are people out there using it.

When Rails 3.1 came out, there was finally a way to properly integrate the code into Rails, so I sat down on a weekend and made some massive changes to REST in Place:

• No more standalone
  Since I don’t think anyone is using it in a different way, REST in Place is now a pure Rails 3.1 gemplugin using the asset pipeline
• Specs
  To facilitate development of new features and refactoring, the app now has a Jasmine spec suite in the testapp. It can be run by visiting http://localhost:3000/jasmine
• CoffeeScript
  Because if you’re not using CoffeScript there’s something wrong with you. It makes JavaScript bearable again.
• Proper CSRF token support
  This was hacky in the past but for a while Rails includes a standard mechanism for providing CSRF information to JavaScript running on the page.
• Automatic detection of include_root_in_json setting in Rails
  Through the magic of erb and the asset pipeline, REST in Place can detect how you’ve configured ActiveRecord.
• BC breaking interface changes
  Two major things have changed that might trip you up:
  • The default css class for REST in Place is now rest-in-place and not rest_in_place.
  • Elements with that default class are initialized automatically on document.ready.
  • The jQuery function to intialize elements with a REST in Place Editor was renamed from rest_in_place() to restInPlace().

All this is now available in a nice gem. All you need to do is add gem 'rest_in_place' to your Gemfile and you’re rollin’ (You might want to take a look into the README first).

Now that I have CoffeeScript and specs, development of new features can be done much faster and easier. Some things I have planned:

• Differentiate between manual abort and a failure
• Validations/proper error responses
• Maybe a nicer set of default widgets and UI improvements
• Rails view helpers

Especially the last two points go a bit against my original intentions for REST in Place, but if it helps more people to use it, hell, why not.

Unfortunately, these changes have come with a very bad timing. One day after I pushed the first beta of 2.0 to github, Ryan Bates did a Railscast about in place editing. While REST in Place is mentioned there, he’s showing Best In Place, a pretty extensive fork of my code that already has most of what I just wrote (asset pipeline, specs) or am going to write (validations, helpers).

Well, as frustrating as that was, I didn’t come as much of a surprise, considered how I’ve neglected REST in Place’s development in the past. I won’t repeat that mistake again, promise.

As much as I love Tumblr’s auto pagination, one of the big gripes I have with it is that after a while of scrolling it becomes absolutely deadly to mistakenly click on a link that makes the browser leave the page. Because all that content was loaded with Ajax, there’s no way of getting back to where you were except to repeat the scrolling orgy again.

The script was working fine but relied on an unpublished callback hook in the Tumblr Dashboard JavaScript code. Because of that, the script did not run as a Safari Extension, since Extensions only have access to a pages DOM but not to its JavaScript scope.

One day Tumblr changed something and the whole thing stopped working. So I rewrote it, this time coding against the DOM and not the JavaScript code. This opened up the possibility to create a Safari Extension and since that script was extremely simple, also a Chrome Extension.

You can download them both here:

• TumblrPushstate Extension for Safari
• TumblrPushstate Extension for Chrome

Enjoy!

Old App vs. New App

This was the Application descriptor for the old version:

This was the Application descriptor for the new version:

Flex 4.1 Apps that use Air require Air 2.0 as a minimum, Flex 4.5 Apps require Air 2.6.

The Air Updater Framework

In the application descriptors above you see that the version tag has changed to versionNumber. This change was introduced in Air 2.5. At the same time, Adobe introduced a similar change to the update descriptors. These are two versions of the same Air update, once for Air < 2.5, once for Air >= 2.5:

The Air update process works like this:

1. The Air updater reads the update descriptor from a server
2. If it detects a version increase, it downloads the new version
3. It unpacks the new version and verifies the application descriptor
4. It installs the new version and restarts the app

The problem is step 3: The Updater from Air 2.0 can’t process the Air 2.6 application descriptor and throws an Error #16824 (Version mismatch).

The solution (theory)

Adobe provides a solution in the knowledge base that is confusing and can be translated to simpler terms:

1. Create an intermediary version that is built with Air < 2.5, but compile it with the Application Updater libraries from Air >= 2.5
2. Distribute that version to your clients using a Air 2.0 update.xml
3. Create the new version with Air >= 2.5
4. Distribute the new version using a Air 2.0 update.xml
5. From then on you should use the 2.5 namespace for the update.xml

Step 4 is crucial:
Although the intermediary version was built with the new update framework that can process the 2.5 update.xml, doing so will result in an error.
Air will complain that the update descriptor namespace must match the application descriptor namespace (of the currently running application, not the one you just downloaded).

Now, all this is quite frustrating but it sounds pretty clear to implement, until you actually start with step 1.

The solution (practice)

This isn’t actually all that hard with Flash Builder:
Go to “Project Properties”, “Build Path”, unfold the Flex Framework in the list and remove applicationupdater.swc and applcationupdater_ui.swc. Then click “Add SWC” and add those two files from your Flex 4.5 framework location (/Applications/Adobe Flash Builder 4.5/sdks/4.5.0/frameworks/libs/air/applicationupdater.swc on Mac).

The catch? All but the most trivial of applications will use the commandline compiler in a custom build script or the mxmlc task in an Ant buildfile. How do you perform the equivalent of what we just did in Flash Builder in those cases?

This crap took me half a day to figure out. I’m giving you the Ant version:

Before, my compile task in my build.xml looked like this:

I had to change it to this:

Now you have four library-path directives:

1. The first library-path is append="false" and thus clears out any existing library paths that have been set through the frameworks default flex-config.xml and air-config.xml. Inside that block, I include the default flex libraries.
2. Next, I include the new updater framework.
3. In the third step, I include the other Air-related files as they would have been by the air-config.xml file. Because the updater framework 2.5 has already been loaded, this line will not include the 2.0 updater.
4. Finally, I add my custom libraries.

The equivalent solution for the mxmlc commandline compiler should be (not tested):

Of course there’s also still the option to subscribe to the comments feed for an individual post. The feed is available in the HTML header and should be picked up by a reader. Below the comment form, you’ll also find a manual link.

As before, I want you to go into a repository of your choice and poke around a bit yourself while you’re reading my explanations.

The .git directory

In your working directory, run cd .git to visit the .git directory. Here, Git stores everything it needs to run: configuration, the database, hooks and refs. I want to explain every subdirectory briefly, before going into the details of the more interesting parts.

The configuration file config

This file stores options you have configured directly via git config or automatically through other commands. For example, git clone populates this file with the default “origin” remote:

You can edit this file in any text editor. This is sometimes easier than using git config.

The uppercase files

You will notice some files in the .git directory that are named in uppercase letters. Let’s keep this brief, I will get into more detail later.

description, hooks and info

The description file contains a description of your repository. You’ll likely never use this unless you plan to publish your repository through gitweb.

The hooks directory contains callback scripts that are executed by git everytime certain event occurs (like a commit or a rebase). These can be used to send out emails everytime someone pushes a commit to a server for example.

Inside the info directory, the only file you’ll probably ever touch is the excludes file, which contains your private excludes. You can use it to prevent temporary files from showing up in git st without adding them to .gitignore.

index and logs

The index is a central mechanism of Git. Basically it contains the content of your next commit. I like to call the index an unborn commit. By adding and removing files through git add and git rm you shape it’s content to your liking and then store it in the database as a proper commit through git commit.

The logs directory contains specials files known as reflogs. Each of the files here corresponds to a branch. Whenever you are working in that branch, an entry is created in the reflog. This makes it possible to see what commit your branch was pointing to, at any given moment in time using git reflog <branchname>. I will talk a bit more about the reflog in the next part of the series.

The objects directory

Now it gets interesting. The objects directory contains the actual database of all the objects in the repository. The objects are stored in files and directories that are based on the objects SHA1 ids. The first two characters of the SHA1 form a directory, the rest is the filename. If you cat any of the files in there, you’ll see the binary contents of the object, compressed with zlib. To see the uncompressed content, use the following command (you’ll obviously need Ruby for this):

Remember what I told you at the end of part one? That git stores all of its objects as actual files? Here you see them. Also, remember that I told you that it didn’t actually do that all of the time? Well, run git gc and list the contents of the objects directory again. Most of the directories should be gone now. They went into one of the files in objects/pack. These are compressed archives that allow for much more efficient storage of the objects. But it helps to still think of them as the actual files we’ve seen before.

The refs directory

The refs directory sits at the interface between the user and the object database. Here, branches and tags are stored, enabling you to access commits by an easy to remember name instead of the SHA1. Inside refs you’ll see several subirectories: heads and remotes store branches for the local and remote repositories respectively, tags contains tags. If you’ve used git bisect or git stash before, you’ll also find corresponding files for them here.

You can take a look at what your refs are pointing to by just looking at their content. They simply store the hash of the object they’re referencing in plain text.

You might be wondering, that git branch -av is showing you quite a lot more branches than you see files in the refs directory. That’s because only branches you’re actually working with are listed here as files. The rest can be found in the file packed-refs in your .git directory.

Working with branches

Now that you know how branches are stored, you can probably imagine how some of Gits common operations are implemented. Lets take a simple commit for example.

• Let’s assume you’re working in the master branch. Your HEAD will point to that branch. Execute a cat .git/HEAD and you’ll see a reference to the master branch:
  
  ref: refs/heads/master
   
  Master itself might point to a commit:
  
  jan@mops$ cat .git/refs/heads/master
  05c80116a36bbbdd7a453255aee5a1d2c7b01fd7
  jan@mops$ git rev-parse master
  05c80116a36bbbdd7a453255aee5a1d2c7b01fd7
   
  HEAD can either point to a branch, like shown, or directly to a commit (That’s called a detached HEAD, a term you might have encountered already). Git has no problems resolving HEAD to a commit in any case:
  
  jan@mops$ git rev-parse HEAD
  05c80116a36bbbdd7a453255aee5a1d2c7b01fd7
   
  This situation is displayed in the illustration.
• Before you start editing, your working tree, your index and the tree object that belongs to the current commit that HEAD points to have identical content. This is situation 1 in the illustration.
• You will now edit a file. The git status command will report that there’s a difference between your working directory and the index and list the file under “Changed but not updated”. This is situation 2 in the illustration.
• After adding our changes to the index with git add, git st will now report difference between the index an the HEAD under “Changes to be committed”. We’re now at situation 3.
• If you’re done with your work, you finally call git commit. Git then takes your index and creates a tree object from it. A commit object is created, containing the commit message, your name and the current time. The commits parent will be set to the commit that is referenced by the current HEAD and its tree reference will point to the tree that was just created. This is the transition from situation 4 to situation 5.
• Finally, to treat that newly created commit as the new tip of your development history, git updates HEAD to point to it. In case HEAD references a branch, the branch is updated. At every step you can see the pointers changing by looking into your HEAD and refs/* files. You’re now at situation 6 and your repository is in a clean state again.

By now, you can probably already imagine how branches are created. Git simply places a file with the name of the branch in refs/heads and lets it point to the commit you provided to git branch.

Checkouts are a little more interesting. If you instruct git to checkout a branch, three things happen:

• The index is set to the same contents as the commit you’re checking out
• The working directory is also adjusted to the same contents
• If you’re checking out an actual branch (as opposed to, say a tag or a SHA1-identified commit), git updates HEAD to point to that branch.

Now you know what the HEAD file I introduced in the “uppercase files” section is used for. Just as HEAD stores the pointer to your current branch, the other uppercase files point to other branches, or other commits that are interesting in some situations like a merge or fetch operation.

Summary

The last part of the series described the data structures behind Gits object database. By discussing the contents of the .git directory, you understand the operations that git performs to organize the content in the object database, and to create branches. Given the knowledge about these files, you should have a clear idea now how Git implements its commands.

In the next part of the series, I want to take a closer look at some of them, especially the dreaded rebase.

This is why I love working at 9elements.

If you used Git before and kinda like it but feel unsure about using some of its advanced commands because you think you don’t completely understand whats going on under Gits hood, if you like what rebase can do for you but are afraid to use it because you’ve read somewhere that the sky will fall on your head if you make a mistake, then this article is for you. Git only reveals its true, awesome power if you use it to its fullest potential. And to do that it is essential to understand how Git works internally.

The talk I gave at the Barcamp was roughly about four topics:

1. Data structures in Git
2. The layout of the .git directory
3. The benefits of using rebase and why rebase isn’t nearly as harmful as everyone thinks
4. Several Tips and Tricks for making day-to-day tasks easier

After the talk I decided to write it down here for the benefit of everyone who ever struggled to understand what Git exactly does when you tell it to pull, merge or commit. I will write a series of four posts, based on the topics of the talk.

Data Structures in Git

Git’s core database is a directed object graph with four different types of objects. Each object has an identifier that is calculated as a SHA1 hash of its contents. That hash is formed by a cryptographic function returning a 128bit key. The fundamental property of a hash function in this context is that it’s a true function in the mathematical sense. Same inputs yield the same outputs. This ensures that identical objects are always assigned to the same identifier. There is no duplication, ever. I will describe this priniciple in more detail in the following paragraphs.

It will help your understanding to refer back to the following graphic when reading these paragraphs: This is a representation of Git’s object graph. For brevity I focused on the structural properties of each object, omitting the content (the binary content in case of a blob, the commit message in commits etc.). I also shortened the SHA1s to three characters.

The four object types in Git are:

Blobs

Blobs are simply chunks of binary data with no other properties, no metadata no nothing. Just the pure data. They are used to store the content of files in the repository. They do not correspond 1:1 to files however. They correspond to file content. Two files in your repository, with different names or at different locations, with identical file content will use the same blob object to represent that content in the database. The blob is identified by the SHA1 hash of its content.

Trees

Trees represent directory structures. This is a tree object:

Where does this representation come from? Well, Git offers a command for that, git cat-file. Its most common usage is git cat-file -p <object>. I got the above printout by passing the SHA1 of a tree as <object>.

You can see that a tree is simply a list of your directory that consists of links to other objects, blobs (for files) and trees (for subdirectories), together with metadata (file permissions and filenames). What this says, for example, is that there’s a file named “Capfile”, whose content is stored in the blob with the SHA1 e04728e8d391f57a6fa0c3325118750c602ef5ef:

Or that “app” is a subdirectory whose content is found in the tree 875c4668c815306dcb1de23407973e2f1fb9d3a8:

Commits

Until now, our trees and blobs have been floating around in the database with no way of getting at any object without knowing its SHA1. Also, we’ve seen the information that blobs and trees can store but there was no discernible way of storing the history of anything. Pretty useless for a version control system, you say?

This is were commits come into play. Their job is to record history. Let’s start by looking at the commit on top of our master branch with git cat-file -p master:

Here you can see what kind of information is stored in a commit:

• An author and a time of authoring as well as a committer and the date the commit was created. This distinction is made because Git supports patches that are authored by one person but committed by someone else, something that’s not uncommon in big open source projects.
  This can also occur when all developers have commit rights: Whenever you cherry-pick a commit, you become the committer, but the original author remains the same. However, for the sake of discussing Git’s data structure this distinction has no relevance.
• A reference to a tree object that represents the state of the working directory at the time the commit was created.
• One or more references to parent commits. This is what actually builds the history of your repository. Each regular commit has one parent, one previous state of the working directory. When you perform a merge, a commit can even have two or more parents, pointing to the different branches of development that have been merged.

Taking the example from above, if we inspect the parent we see such a merge commit:

This explains how Git strings a series of commits together to form a history, but I didn’t tell how to actually get at an object without knowing its SHA1. This is were branches come into play. They are essentially readable aliases for SHA1s that get updated every time you perform certain actions (like committing, merging, etc.). I will explain this in more detail in the next post of this series.

Tags

The last type of objects are tags. To be more specific, Annotated Tags. Simple tags are not objects (more on that later), but annotated tags are. You get an annotated tag if you use the -a option when creating a tag

A tag consists of an object reference, but what’s special about it, is that it can refer to any kind of object. What exactly the tag is referring to, can be seen in the type field. The tag we’re seeing here points to a commit with the SHA1 4a06c46ee6d58ce4be09954ee054921b18269cd6. The tag also has a name, given in the tag field, a tagger and a message.

To be honest I never worked with annotated tags and most of you probably never will either. Their main use case over regular tags is that they can be cryptographically signed (as can commits).

Summary

That was it. Four very simple types of objects.

• Blobs – Storing the content of files
• Trees – Storing the structure of your working directory
• Commits – Putting trees into a sequence to preserve history
• Tags – Reliable mechnism to point to objects in the database

Maybe you should take a look at one of your own repositories now, starting with git cat-file -p master and poking around a bit.

These objects and their references to each other are the absolute core of Git and understanding their structure and relationships is essential to working well with Git. As soon as you start thinking of your repository as this objectgraph, you’ll realize that the Git toolchain is nothing but a set of manipulations on that graph database, creating new objects all the time, pointing to other objects.

There are two additional implementation details that you should be aware of:

1. Even though my description and the results of git cat-file make it seem like you’re dealing with full fledged objects, the reality is that Git uses very efficient compression algorithms to reduce the amount of actual data stored in its database. Trees or blobs aren’t usually stored in full but described as differences to other similar objects.
   But for reasoning about objects, you can and should think of them as being self-contained and independent.
2. On the other hand, objects often actually are uncompressed in the database. Also Git never deletes objects! If you remove a file from a tree, the blob for that file will still exist. If you lose a commit through rebasing or merge problems or by accidentally deleting a branch, as long as you know its SHA1, you can get back to it. The only time Git actually deletes and compresses objects is during its garbage collection run. If you clone a new project, you will retrieve compressed objects from the remote server, but objects you create in your local repository will be uncompressed at first. Git repacks and garbage collects on its own from time to time, but you can also trigger this processs manually by calling git gc. Do this when you notice that working with your repository becomes slow or that the repository becomes too large.

Next post

In the next post of this series, I will explain the structure of the .git directory. This is where you will find your branches and regular tags, as well as the actual object database files. You will learn what a branch actually is and how to effectively manipulate them.

Dueck spricht darin im Grunde von demselben Thema, der Überflüssigwerdung bestimmter Berufe. Die Argumentation in meinem Blogpost war, dass all die Berufe verschwinden, deren Existenz sich aus Beherrschung von Technologien rechtfertigt, die langsam von jedermann beherrschbar werden.

Dueck geht noch einen Schritt weiter: Seiner Ansicht nach verschwinden zu einem großen Teil ausserdem die Berufe, deren Existenz sich aus der Beherrschung von Wissen rechtfertigt. Wenn alles Wissen jederzeit für jeden im Internet zugänglich ist, dann ist die Beherrschung von Wissen kein Monopol mehr, so wie heute bereits die Beherrschung von Technologie kein Monopol mehr darstellt.

Es wird in dem Vortrag teilweise stark vereinfacht und übertrieben, aber der Grundgedanke ist nicht von der Hand zu weisen. Dueck folgert hieraus eine kontroverse These: Wichtigste Qualifikation werden in Zukunft ausschließlich Softskills sein. Führungsqualitäten, Einfühlungsvermögen, Begeisterung, Teamfähigkeit. Dinge, die man nicht nachlesen oder auf die Schnelle lernen kann. Und auf diese Entwicklung ist die Gesellschaft nicht vorbereitet.

Ein hervorragender Vortrag:

Download/View Slides

The talk has also been recorded by Oliver Überholz, who promised to send me a copy but so far hasn’t replied to my messages. Please give him a nudge :)

UPDATE: First post of of the Advanced Git series is online.

history.replaceState to the rescue!

I submitted this as a suggestion to tumblr:

It often happens to me that I scroll my dashboard for a really long time, then click a link and forget to hold CMD to open it in the background. The entire scroll state is lost then when I get back to the dashboard. I think it would be a good idea to use the pushstate api to update the locationbar in a way similar to what happens if I click reblog on something. After I’ve sent the reblog, it takes me back to the correct place in my dashboard. Using pushstate would make this work even with external links.

This would also solve another problem: Sometimes, the loading indicator at the bottom of the page is stuck. In a normal webpage I would hit stop and reload and get my content but there’s no way to do that here because then I have to scroll aaaaaall the way down again. With my approximate location stored in the url bar, this wouldn’t be a problem anymore.

I think these features would also help you reduce the load on your servers a little because people wouldn’t need to scroll over content they’ve already read as much anymore. Browsers not supporting pushstate could just keep working in the current way.

Since they did not reply yet, I decided to implement this on my own. I wrote a tiny GreaseKit script that does exactly what the suggestion said. You can install it here.

This will also work in GreaseMonkey. I might make a Safari Extension from this later.

UPDATE:

I probably won’t make a Safari Extension. Extensions run in a Sandbox that makes it impossible to interact with the JavaScript from Tumblr in the way that my solution does. All I can do is to manipulate the DOM.
Maybe I could solve this by watching the DOM for changes (as opposed to using Tumblr’s handy callback hook) to manipulate the history but that seems kinda cumbersome right now.

UPDATE 2:

I turned this into a Safari and Chrome Extension.