Mercurial workflow by Tor Norbye

Mercurial Tip: Checking in regularly with uncommitted changes in your clone

(Oracle is being a f-- all by moving blogs from the sun servers or what have you. Using google's cache to post this here)

(Disclaimer, this was not written by me but I am using it and want it handy to give to other developers of my team)

NetBeans, like OpenJDK and OpenSolaris, uses the Mercurial distributed version control system. I'm a big fan of of distributed version control. However, one thing that drives me nuts is this error message:

% hg merge abort: outstanding uncommitted changes

This isn't just going to be a rant - I've finally found a solution which is working extremely well for me. I've suggested it to some other developers who have also reported that it works well for them, so I thought I would share it with you.

The reason I run into this all the time is my preferred style of work:

  • I like to work on many things simultaneously. If while working on something, I come across a bug and I spot the problem, I just fix it right there and add a test for it. When I get ideas, I might go and put @todo tasks for myself, or leave editorial comments in various source files. I might also work on a couple of larger tasks simultaneously. Yes, I know the preferred Mercurial idiom for this is to have multiple clones, one for each task - but that isn't my preferred way of doing it. Each NetBeans clone is huge and takes a while to both clone and build - and I would have to switch my IDE editing context between these large source trees all the time.
  • I like to check things in in logical chunks with a message that pertains to that particular change. This means I often check in a subset of the edited files in the tree.
  • I like to check in code regularly rather than leaving all files edited until I'm ready to check everything in.
  • I like to keep up to date. I'm using the bits I'm working on and I report problems when I see them so I want to make sure these are truly new and current bugs. Therefore, I update to the tip at least every day.

It turns out that the above requirements are tricky to fulfill with a default setup. Here's what happens.

  • I'm working on something and a high priority bug comes in. I find and fix it. I check in just the fix for this one problem. I still have many other edited files in my clone since I was in the middle of something else.

  • Since it's a high priority bug, I want to push it. However, when I do, I get this: 
    % hg push
    pushing to https://tor:***@hg.netbeans.org/main/
    searching for changes
    abort: push creates new remote heads!
    (did you forget to merge? use push -f to force)
    This tells me that somebody has checked in something since my last pull. That happens a lot with a project like NetBeans where there are checkins every few minutes.

  • No problem - I'll just pull down their changes, merge, and push again, right? 
    % hg pull && hg update
    pulling from http://hg.netbeans.org/main/
    searching for changes
    adding changesets
    adding manifests
    adding file changes
    added 1 changesets with 1 changes to 1 files (+1 heads)
    (run 'hg heads' to see heads, 'hg merge' to merge)
    abort: crosses branches (use 'hg merge' or 'hg update -C')
  • Ah, I couldn't just update, I have to merge since I have checked in changesets of my own. Let's do that: 
    % hg merge
    abort: outstanding uncommitted changes
  • !@#@%*#%$%&*$ ! Now, if I have dozens of changes, what do I do? When I first started using Mercurial, I would either try to finish my other changes and check them in and then try the merge again, or if I really wasn't ready to do that, I would copy the contents of my modified files to another location, hg revert the changes, and try again.
    1. As before, work in main. Edit files at will, and check in logical changesets as needed.

    2. When you want to push a changeset, go to your sync clone, and pull your new changesets from your working clone: 
      % cd ../sync
      % hg pull ../main && hg update
    3. At this point, it's best to try a build/test as well, to make sure changeset is complete. If you're working on many things in parallel, it's possible that your changeset is depending on a change in a file you haven't checked in yet.

    4. Now pull in the new changes from the master, and merge these: 
      % hg fetch
      (Or if don't have the fetch extension installed, do it manually - hg pull && hg merge && hg ci -m "Merge").

    5. If something went wrong with the merge - no problem. You can just go and nuke the entire sync clone and try again! Your modified files, and your new changesets, are still sitting completely unaffected in the main clone. Just clone sync again and try more carefully :)

    6. Now you can push your merged changesets to the master repository: 
      % hg push
    7. ...and now you can pull these changes back into your main repository: 
      % hg pull ../sync && hg update
      This will give you all the tip changes into your working clone, but without the risk of causing multiple heads that you have to merge. You've already merged your local changesets over in the sync clone, and therefore there is no conflict between your local changesets and the new changesets in the tip!

    • It's not unreasonable of Mercurial to insist that you cannot merge with uncommitted changes. When it's merging, it will think file modifications in the clone are your manual edits to resolve any merge conflicts. However, it does make using my workflow difficult.

    The Solution

    It's really very simple: Use two clones. Do all your work in clone "main", and all your merging and pushing from clone "sync". Let's say the main repository is "master", and your two clones are "main" and "sync":

    To get set up, either clone the master repository twice, or, you can clone your main locally, but be sure to update the "parent" reference in your sync clone such that it points to the master instead of your working clone (I do that by copying main/.hg/hgrc to sync/.hg/hgrc.)

    Here's the new workflow:

    This process may seem tricky, but it's trivial once you try it:
    Work in main, push and merge in sync, and pull back into main.
    Finally, I want to call attention to item #4. Doing it this way means that it's trivial to try again if something wrong happens during the merge. I've had a couple of merges where I've really mucked things up. Unfortunately, this was in my tree that contained the changesets that I cared about. In the end I had to go and manually copy out the files I wanted to check in and try again. With the above approach, if something goes wrong, just nuke the sync clone and try again.

    This is the reason I'm suggesting this approach to anyone using Mercurial, not just people who want to work with edited files. Especially when you're new to distributed version control systems or Mercurial, it's great to be able to go back if you make a mistake. Just make sure you know what you're doing before you submit that final hg push command to push everything back to the master repository!

Happy B'day mate!

Hey Faiyaz,

Happy 30th. How does it feel? I'll raise a toast to more maturity and immaturity that goes with the territory ;)

A friend of mine once was telling me: His aunt who never got married was once visiting him in NYC. She was in her mid-forties. She said, she feels the same age/maturity level at the time she was 25. Only the body responds a little slower. Maybe a little extra pounds here and there. But overall, nothing felt changed.
It was a curious story. Instructive perhaps. You need to get married and have kids to feel old.

Cheers!
Archit

Default download link should be a torrent

I am downloading Lubuntu (a lightweight version of Ubuntu). The torrent download is going at 240KB/s = 1.8 Mbps. My ISP gives me an advertised .5Mbps. So this sweetness is courtesy the 80 seeders of Lubuntu out there.

Well I guess in this day and age, the only monolithic things you download are the OS, the browser and the IDE. Aah, well. Be kind to your favorite Linux hosting sites. Use a torrent. They weren't built purely for illegal music you know.

-----------------------------------

As an aside, here are my reasons to install Linux: I am sick of Windows, I am trying to learn Lisp and the uber cool crowd says I need emacs, I am going to play with either Jekyll or Hyde, so I need either RoR or Django. Good good.

----------------------------------

[EDIT] Further update on the install experience: VMware workstation 6. Lubuntu. Install is a breeze and I have a SSD so I don't feel the disk I/O crunch. Installing VMware tools. Now my question is: How the hell does a linux distro ship without GCC? To quote Mcgrof: Oh Lords of Kobol! Installing gcc-4.1 I feel brand new. Last compiler I worked with was gcc3.2.

The distro did come with Python. No ruby. Aah well, everything's an apt-get away (until it isn't). Its good to be back. Hopefully no major time sucks.

The Latur earthquake and choices we need to make

In 1993, an earthquake struck Latur in Mahrashtra, in the western part of India. The loss of life was massive due to the poor construction materials used to build houses. I must have been 9 years old at the time.

Around that time, the school had organized a school picnic to take the kids to a city nearby Delhi. I can't remember but I vaguely think it was Meerut. The picnic was optional as you had to pay Rs. 370. It was a large sum of money in 1993, it was almost a month's school tuition. As the news of the earthquake came in, the school started a fund raising campaign, asking the children to request their parents to donate money.

I went home told my dad about both things in the evening. He offered a choice, to either go to the picnic or donate the money to the victims of the earthquake. A nine year old presented with such a choice has a very tough decision to make. Fortunately, I chose to donate the money. My dad gave me the cash that very moment to take to the school.

I went to school the next day, went up to the class teacher to give that money to the teacher. She counted it and said, oh so this is for the picnic. No, I said this is for the Latur earthquake fund. She understood. She mentioned it to the class. That was it. I don't think it prompted any other kids to follow the example.

Why am I writing about this today?

I see poverty, stark poverty around me, all day, every day. Food is a luxury, clothing is a luxury. Everything appears to be a luxury. Any consumption seems to be wasteful, extravagant. The question which strikes me is how many people could I have educated with any money I spend on any kind of consumption. The luxury of living in the first world is that you don't see the poverty. It is on TV, on hulu advertisements. Not a real, breathing 9 year old girl who I saw today sweeping the floors. What is the promise of the city, the country, the world to this nine year old? I am sorry but Gandhiji's talisman which was printed on all NCERT books from grade school to high school, which was promptly ignored, seems relevant today. Here it is:

"I will give you a talisman. Whenever you are in doubt, or when the self becomes too much with you, apply the following test. Recall the face of the poorest and the weakest man [woman] whom you may have seen, and ask yourself, if the step you contemplate is going to be of any use to him [her]. Will he [she] gain anything by it? Will it restore him [her] to a control over his [her] own life and destiny? In other words, will it lead to swaraj [freedom] for the hungry and spiritually starving millions?
Then you will find your doubts and your self melt away."

- One of the last notes left behind by Gandhi in 1948

What will we do for the swaraj of this nine year old? We need to figure out how we will educate the young, so that they can be overfed little buggers who are annoying in their adidas sneakers.

In this education, mind you, we have to tell our little girl that she needs to save the earth, her rivers, her trees, her mountains. We have to teach her algebra, her local language, English and using a computer. Maybe she'll learn how to program, maybe she'll be an artist, maybe a musician.

I'll start work on this. I am trying to think what can I do which the hacker in me wants to scale.

I can think of the fantastic work of Khan Academy. I can think of translating the primary and middle school lectures to Hindi.

I also came across Teach for India. Its a fantastic program. A two year commitment. No sitting on the fence, no postponing, no excuse. Its a real startup. Its fantastic. I don't want to proselytize. I'll try to figure out a way to help them while I work on building my startup.

A startup survey, a raffle and an iPad

I am raffling away an iPad in exchange for taking a 5 question survey
on how you do your taxes.  It will take three minutes to fill out and
the chances of winning are very good. If you are interested, please
click here:  https://spreadsheets.google.com/viewform?formkey=dFZuQ0xBUHl2Ny1wQ2dZenBoNGpoMHc6MQ

Additionally we are raffling Rs. 500 for every 100 people who respond
to the survey.

We are a stealth mode startup located in Delhi, composed of myself and
three others.  The company is focused on online tax preparation and
filing.  One way that we are validating our technology and solution is
through this survey.

It's rare that I send out broadcast messages - we are trying to build
a new company where friends and family input is very valuable to us.
I thank you for your help!

Cheers,
Archit
http://goo.gl/MCMdH

D.Light and social entrepreneurship

India has a way of making you think of social entrepreneurship. I am getting slowly nudged into thinking about it more clearly than before.

Just came across D.Light: Makers of solar powered lanterns for $10 (Rs 450). The key idea is to replace kerosene lamps which are unsafe and have a high associated running cost. User stories from rural India report the products are perceived as absolute godsend. 

Looking just at the website and other reports, the key insight for me are two-fold. 

* The message: The company's mission and message is very clear. Social good is the aim. Economics are not ignored. Once you look, the product pricing and features/key benefits are presented upfront just like a product company.

* The fellowship program: http://dlightdesign.com/fellowship_program.php Most young, smart people might not be looking for a job when they want to figure out their next dig say after college or after their first corporate stint. Very sweet. An idea I'll try to emulate.

My mother's compassion.

I want to always remember the day of January 29th 2011.

In India it is easy to get inured at the plight of the poor. It is simple to look away and not care. My mother showed me otherwise. She has shown compassion on two different occasions. One more dramatic and many years ago which I'll write about later. This one is a more simple, a more humble incident.

An old man was collecting a mound of dirt near our house. He'd take the dirt in a plastic bag and then walk quickly to throw it away at a corner a 100 yards away. He was a frail man barely able to lift the bag of dirt. It was freezing cold at 6:30 AM in the morning. He was wearing a thin kurta pajama, has no shoes on. He clearly was of poor mental health.

My mother when she saw him, gave him an old pair of shoes and then ran in to fetch a sweater. She asked me to go out to give it to him. I had just woken up, I sleepily waved her off. She went out and gave the sweater to the guy. He continued his dirt collection after wearing the sweater.

My mother's action aroused deep feeling in me as it was easy to ignore this man. The poor, the downtrodden. I wouldn't have given him a moment's notice thinking there's too much in India to fix.
Why do I keep mentioning the whole of India? There's a backstory. I have just returned from the US after living there for 5 years. I am starting my own business. To be an entrepreneur. And I just realized my own stupidity. That every day I go to work to move the needle just an inch and I am happy if it budges an inch.
There's change needed for the country to be a better place, for the city, for the neighborhood to be a better place. I'll have to focus on moving this needle just an inch too.

I also had a fleeting desire to photograph the man with my shiny iPhone. I did not. I realized the irony of the situation. I don't want the world to see the poverty porn either. No writing a quippy twitter post for the entertainment of random folks.

The man went away after a while. Ideally I should have tried to figure out how he survives, lives etc. But I didn't. I'll remember that too. I didn't try to do anything.