debaira: Newbie: Howto Improve Your Productivity With Emacs

10 Specific Ways to Improve Your Productivity With Emacs

Emacs is the world's best text editor. It's not just the best for editing program source; it's the best for any kind of text-editing. Mastering Emacs will make you more effective at writing and editing email, documentation drafts, blogs, HTML pages, XML files, and virtually everything else that requires any typing.

The tips in this little document are geared towards Emacs power-users. You should be familiar with the basics of launching and editing with Emacs, and you should already know the essentials of copying stuff into your .emacs file, and debugging things (or finding a friendly Emacs Wizard) when something goes wrong.

Not all the tips are customizations to Emacs; some of them are changes to your desktop environment to make it work more seamlessly with Emacs.

The key to understanding Emacs is that it's all about efficiency, which includes economy of motion. Any trained musician will tell you that economy of motion is critical to becoming a world-class virtuoso. Any unnecessary motion is wasted energy and yields sloppy results.

Using the mouse is almost always the worst possible violation of economy of motion, because you have to pick your hand up and fumble around with it. The mouse is a clumsy instrument, and Emacs gurus consider it a cache miss when they have to resort to using it.

Compared to Emacs Wizards, graphical-IDE users are the equivalent of amateur musicians, pawing at their instrument with a sort of desperation. An IDE has blinking lights and pretty dialogs that you can't interact with properly (see Item 6), and gives newbies a nice comfortable sense of control. But that control is extremely crude, and all serious programmers prefer something that gives them more power.

IDEs also offer Refactoring tools, which are all the rage, because they help you automatically fix your screwed-up code. This is fine, as far as it goes, but I'll tell you a secret: Refactoring tools don't understand English. They won't help you one bit when you're doing something that the tools weren't designed to handle. Refactoring tools are cookie-cutter solutions; Emacs offers you a level of fine-grained control over your environment that makes you the equal of any editing challenge.

However, as I often point out, this has to be seen to be believed, and even if you believe it, you need to make a serious, lifelong commitment to Emacs in order to master it. Mastering it involves learning Lisp and customizing Emacs to the finest degree imaginable. This simply serves to increase your hunger for more control and more automation, so even if you've mastered Emacs, you're never really finished extending it.

So it's not an editor for the faint of heart, and this blog is targeted at people who have already made the commitment, and want to improve their mastery of this elegant, timeless piece of software.

The rest of you: I think your Eclipse just finished launching, so you can get back to work now.

Item 1: Swap Caps-Lock and Control

On Windows and Mac keyboards, the Ctrl key is awkwardly located in the far lower-left position on the keyboard. The Ctrl key is critical to using Emacs at all times, so you'll never become an Emacs virtuoso unless you move it to an easier position. That position should be on Home Row, so Caps Lock is the best choice. That's the location of the Control key on most Unix workstation keyboards, for precisely that reason.

To do this on Windows 2000 or XP requires some registry hacking. From the Start menu, choose Run and enter "regedit". In the left-side tree view, click down to:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Keyboard Layout

Click on the KeyboardLayout entry to give it the focus. Make sure it has the focus and not one of its children. Then from the Edit menu, choose New Binary Value, and name it Scancode Map. It should show as type REG_BINARY.

Then select the new Scancode Map entry you just created, and from the Edit menu (whose contents should have changed), choose Modify Binary Data. In the dialog box called Edit Binary Value, enter the following data:

 0000: 00 00 00 00 00 00 00 00
0008: 03 00 00 00 3A 00 1D 00
0010: 1D 00 3A 00 00 00 00 00

Select OK to close the dialog, then exit the Registry Editor. The caps and ctrl keys should be swapped as soon as you log out and back in again. It may require a reboot.

On Linux in X-Windows, you use the xmodmap utility. Create a file in your home directory called .xmodmap if it doesn't already exist, and put in the following lines:

 !
! Swap Caps_Lock and Control_L
!
remove Lock = Caps_Lock
remove Control = Control_L
keysym Control_L = Caps_Lock
keysym Caps_Lock = Control_L
add Lock = Caps_Lock
add Control = Control_L

Save it, and add the line xmodmap ~/.xmodmap into your ~/.bash_profile.

On Mac OS X (Panther and Jaguar) you need to install a modified keyboard driver, which is a little scary, but it seems to work. Here's a discussion of the driver. Alternately, if you're not using a Mac laptop, there appears to be an XML file you can edit as root; it's described here.

This URL has some information on doing it on other systems.

Item 2: Invoke M-x without the Alt key

Alt-x is one of the most frequently typed Emacs key combos, and it involves scrunching your left hand up. Anything you're going to do thousands of times should be streamlined, so you want to be able to start a M-x sequence with the Ctrl key (once you've completed Item 1!)

There's another very important reason to get in the habit of using a Ctrl sequence: the Alt key is unreliable and nonstandard across platforms. In particular, when you're logged in to a remote host via telnet or ssh, Alt-x may or may not work, depending on the system type and the terminal configuration. Rather than mess with the headache of learning to configure every system you work on to know about the Alt key, it's easier to use a key sequence that always works.

The key sequence I use is Ctrl-x Ctrl-m. Note that when you invoke a 2-key sequence with the same modifier key, you can just hold down the modifier key, then press the 2 keys. So with this key sequence, invoking M-x involves pressing and holding Ctrl, typing x, then typing m.

To enable the Ctrl-x Ctrl-m sequence add the following lines to your .emacs file:

(global-set-key "\C-x\C-m" 'execute-extended-command)
(global-set-key "\C-c\C-m" 'execute-extended-command)

I add the second line so that Ctrl-c Ctrl-m will also invoke it, which makes the key sequence even more forgiving. If I miss Ctrl-x and hit Ctrl-c accidentally, it still works. Neither of these key sequences is bound by default in Emacs, so you're not biffing any other useful shortcuts by doing this.

You should practice this key sequence until you get used to it, and then you'll almost never need to use Alt-x again. (You'll still use the Alt key for other commands, which I'll cover later.)

Incidentally, if you want to fine-tune this tip to extraordinary levels, then you probably don't want to use your ring-finger for typing the x-key when you hit Ctrl-x. I use my left index finger, since I'm used to it that way, but you're probably better off using your left middle finger. The reason is that your hand isn't technically on home row when your left pinkie is jammed down on the Caps-Lock key that you've turned into Ctrl. The point is to use whatever requires the least amount of stretching, followed by the least amount of finger motion. You should experiment until you find what's most comfortable for you.

Item 3: Prefer backward-kill-word over Backspace

Emacs Wizards try to avoid hitting the backspace key, because it's just remote enough from home-row to be annoying. We make typing mistakes all the time, but if you type faster than about 50 wpm, it's more economical to kill the entire word and re-type it than to painstakingly backspace to your error.

Here's what you add to your .emacs file:

(global-set-key "\C-w" 'backward-kill-word)
(global-set-key "\C-x\C-k" 'kill-region)
(global-set-key "\C-c\C-k" 'kill-region)

Note that because Ctrl-w was already bound to kill-region, a very important command, you need to re-bind something else to kill-region. I chose Ctrl-x Ctrl-k (and its sloppiness-forgiving companion, Ctrl-c Ctrl-k), primarily because that's the way they did it at my old company, which was filled with wise Emacs Wizards who taught me a lot of what I know. Rebinding Ctrl-x Ctrl-k means it's no longer available for edit-kbd-macro, but you'll use that infrequently enough that it's not something you'll miss having a shortcut for.

As an added benefit, many Unix command shells provide Emacs-like command-line editing keys, and Ctrl-w is usually the default binding for backward-kill-word. So your usage will be consistent in shells.

The faster you type, the more valuable this tip becomes. You'll gradually develop a "feel" for the fastest way to back up and correct various kinds of typing mistakes. In general, here's how my fingers decide these days:

If I've mis-typed a character or two in a fairly long word, and my cursor is still located right after the typo, then I'll use the backspace key.
If the typo is somewhere in the last 15 to 20 characters or so, then I'll usually backward-kill-word to kill backward over the typo, and re-type the words.
If the typo is further back than that, but still on the same line, I'll use Alt-B to skip the cursor back to the word with the typo, and Ctrl-b to move it within the word to get to the typo.

For mistakes further away, I'll use Fast Navigation to get there: Item 4 covers at least part of this technique.

One thing you'll need to be very careful of if you use Ctrl-w for backward-kill-word: Ctrl-w is hardwired to kill the window in many Windows applications. Do not pass go, do not collect $200, and in browser windows, do not save the contents of your form fields. That means if you're typing out something into an HTML form field, and you accidentally make a typo and hit Ctrl-w, kapow! All your work will be lost instantly for you by Good Ole Microsoft Windows. I know of no way to override this horrid behavior. If you know, please tell me.

Item 4: Use incremental search for Navigation

Moving the cursor around efficiently is one of the keys to becoming an Emacs Wizard. IDE users spend most of their time fumbling around with the mouse. They wouldn't dream of doing it any other way, but they don't realize how inefficient their motions are. In the hands of a Master, Emacs becomes the most powerful and efficient text-editing tool on the planet, in large part because it allows you to do almost everything without using the mouse.

Emacs Wizards always have their Emacs sessions as tall as possible, filling the screen vertically, because vertical screen space is such a premium when you're viewing a document. When you can view a fair number of lines of text on screen at once, using incremental search is often much faster than any other way of precisely positioning the cursor.

Get in the habit of using Ctrl-r (isearch-backward) and Ctrl-s (isearch-forward) for moving around in the document. Whenever you need to jump the cursor backward or forward more than about 5 lines, and you can see the target location, you should be using i-search.

To do it effectively, you don't necessarily need to search for the exact word where you want to put the cursor. Let your eye defocus slightly and take in the whole paragraph or region around the target point, and choose a word that looks reasonably unique or easy to type. Then i-search for it to navigate to it. You may need to hit Ctrl-r or Ctrl-s repeatedly if your anchor word turns out not to be unique. But Emacs will highlight all the matches, so if there are more than a couple of them, Ctrl-g out of the search and choose another anchor word.

It's difficult to overemphasize how powerful this technique is, once you've mastered it. Mastering it simply requires that you do it repeatedly until your fingers do it "automatically". Emacs eventually becomes like an extension of your body, and you'll be performing hundreds of different keystrokes and mini-techniques like this one without thinking about them. It's comparable to the hundreds of subtle techniques you acquire for driving a car well.

Item 5: Use Temp Buffers

One of the most powerful things about Emacs is its ability to swiftly generate a new buffer that isn't associated with a file or process. Once you get used to using this technique, you'll sorely miss the functionality in other editors and applications.

To create a temp buffer, just switch to it! Ctrl-x b invokes the command switch-to-buffer, and you just type in adsflj or whatever comes out of drumming on the keyboard. Instantly you've got yourself a scratchpad, where you can take notes, dump temporary results, or use it in any way that's convenient for the problem at hand.

If you plan on keeping multiple temp buffers around, you might give them more memorable names, such as foo, bar, baz, and buh.

If you want to see your temp buffer side-by-side with another buffer, you can split the screen horizontally or vertically. See Item 6.

Because your temp buffers aren't associated with a file, you can kill them just as quickly as you created them using Ctrl-x k, the kill-buffer command.

If you decide you want to save the contents of a temp buffer somewhere, just switch to the buffer and Ctrl-x Ctrl-w to invoke the write-file command. It will prompt you for a filename, and after saving, you can kill the buffer safely and revisit the file contents later.

Item 6: Master the buffer and window commands

You'll frequently want to do some editing task and have multiple windows open. Emacs uses slightly different terminology from most other applications. A "buffer" is a logical space that contains some text, possibly tied to a file or process. A "window" is a visible region on screen displaying exactly one buffer (or part of it). A "frame" is what you call a "window" in OS-lingo: a standalone window with its own title bar and so on.

The most important commands to master are:

Ctrl-x 2: split-window-vertically -- splits your current window into two equal-height windows showing the same buffer (until you change one of them to show something else.)
Ctrl-x 3: split-window-horizontally -- most people don't use this as often, but it's occasionally useful. Splits the window into a left-side and a right-side.
Ctrl-x +: balance-windows -- makes all visible windows approximately equal height. This is useful if you've just done Ctrl-x 2 twice in a row, because you'll have two 1/4-height windows and one 1/2-height window. Ctrl-x + makes them all the same height.
Ctrl-x o: other-window -- moves the cursor into the next window in the window list, which usually means moving it to the window below the current one, or wrapping back up to the top.
Ctrl-x 1: delete-other-windows -- makes the currently focused window fill the entire frame; the others go away. Note that the buffers they were visiting stay around in the buffer-list, so it's always perfectly safe to execute this command.
Ctrl-x Ctrl-b: list-buffers -- shows a list of all the buffers you have open, in a nicely formatted buffer called "*Buffer List*". This buffer's mode has many convenience keys for manipulating the buffer list. For instance, typing "d" while the cursor is on one of the entries will flag that buffer for deletion, "x" will kill the buffer, and so on. Use M-x describe-bindings to view all the Buffer-menu key bindings.

Dialog Boxes: The Root of All Evil

Emacs is an incredibly powerful editor, but a few design choices in particular really stand out as being the biggest individual contributors. One of them is the fact that Emacs has no dialog boxes. This was actually a requirement in order to give Emacs its full functionality while running in a text-only terminal window. But by happy accident, it's also one of the key features that helps make Emacs so insanely powerful.

Dialog boxes suck. For starters, they always have focus issues, and often cause poorly-designed applications to lock up or fail to refresh while the dialog is open. And dialog boxes never seem to play along with any customizations to your video mode. For example, if you set up a dual-monitor display using 2 cards, the application dialogs in Windows will tend to pop up in the wrong window, making the whole experience a really annoying pain in the ass.

Dialogs sometimes come up in unpredictable places even on single-monitor machines. And even in well-designed applications like the Microsoft Office suite, modal dialogs can still wind up buried behind a bunch of other windows on your screen, which makes the application seem like it's totally unresponsive until you find the rogue dialog and bring it to the front.

For some strange reason, dialog boxes are often non-resizable. It's the opposite for application windows: those are almost always resizable, and app designers usually take great pains to make the UI rearrange to fill the space nicely if you resize the window. But with dialogs, the default seems to be non-resizable, possibly because on most OSes, dialogs are a screwed-up hack that was added on long after the window manager was designed without dialogs in mind. (Guess how I know this.) Hell, they're even a mess in Java Swing.

And don't get me started on the buttons. We've had dialog boxes in GUIs for at least 25 years, but people still can't agree on a standard set of buttons for them, or even a standard place to put the buttons. Some dialogs put them in the title bar, some on the bottom, some on the right side. And it's rarely 100% clear what will happen if you use the Window controls in the title bar to close a dialog, bypassing its button choices. The whole experience is a giant crap sandwich, and everyone knows it intuitively, but it's just how everyone assumes things have to be.

But the problem with dialog boxes goes even deeper than the focus, sizing and positioning problems. Dialogs are never, ever full peers of the rest of the application UI. If you define any keyboard macros (not just in Emacs -- in any app, such as Excel or Word), they won't work in dialog boxes. If the dialog has a scrollable widget, you have no options for navigating other than by using the scrollbar.

To illustrate, fire up Internet Explorer and choose Internet Options from the Tools menu. Go to the Advanced tab. There they are: all your pathetic global customization options for IE. If you want to find a particular option, you have to scroll slowly, looking for it. You can't use Edit/Find because the dialog is, of course, modal. And the dialog is, of course, also non-resizable. Most dialogs are like this.

Buffers to the Rescue

In Emacs, all output goes into buffers, and all buffers are first-class citizens. All your favorite navigation shortcuts are available in all buffers, including i-search. You can select and copy text out of any buffer, and there is no such thing as a "modal" buffer, so you can keep the output around as long as you like, while continuing to work in other windows. It doesn't even have to be visible; the "dialog" output stays in the buffer list until you dismiss it.

There is nothing else quite like the Emacs buffer experience in all of application-dom. Once you realize how consistent and powerful this model is, all other applications become slightly distasteful, because you know their UI is just getting in your way.

Once you learn to master those buffers and windows, and sling them around with ease, you'll be on your way to Emacs Wizardhood.

Item 7: Lose the UI

You don't need a menu bar. It's just a crutch placed there for disoriented newbies. You also don't need a toolbar with big happy icons, nor do you need a scrollbar. All of these things are for losers, and they are just taking up precious screen real-estate. Turn them all off with the following code in your .emacs file:

(if (fboundp 'scroll-bar-mode) (scroll-bar-mode -1))
(if (fboundp 'tool-bar-mode) (tool-bar-mode -1))
(if (fboundp 'menu-bar-mode) (menu-bar-mode -1))

You won't miss any of it. We'll cover how to find your way around in the next tip.

(Notes added Jan 2nd, 2006) I recently saw a comment from a person who hated this entire essay because of tip #7. The person expressed a great deal of disgruntlement, evidently being quite attached to his or her mouse and menus, and took affront to being called a "disoriented newbie". The person went on to claim that using the mouse has been proven "faster" by countless studies. So I figure I should elaborate a bit. The remainder of this tip is All New, thanks to that disgruntled reader's blog.

First I should observe that I often wish Emacs had a richer rendering engine, enabling it to do GUI and graphics on par with other desktop applications. It may never happen; my blog-rant The Emacs Problem talks about this a bit. But I'd love to see it. I bring this up as evidence that I'm not a thoughtless anti-GUI person.

Scrollbar: optional

I usually turn off the scrollbar in Emacs because there are keystrokes that can achieve the same effect. However, scrollbars have the advantage of providing analog feedback as to how far into the buffer you are, and how long it is. The digital feedback provided by the %-indicator in the status area isn't as easy to read -- countless studies have proven that. It's why the U.S. Navy uses analog gauges in their reactor plants, for instance. It's too easy to glance at a digital (i.e. numeric) gauge and misread it.

So I have no real problem with scrollbars, if you're more comfortable with them. Just be aware that they'll tempt you to reach for the mouse, but for certain operations (e.g. jumping to the beginning or end of the buffer), it's much faster to use the keyboard. No user studies are necessary to justify this claim, either. Some simple timing experiments should convince even the most skeptical reader.

Keyboard wins for navigation

Let's say we want to put a row of 80 hyphens at the very beginning and very end of a long buffer, and you're currently in the middle of the buffer. It's slightly contrived, but I've done it when putting together a "cut here" excerpt containing the buffer contents. Using the keyboard, I'm done in under 3 seconds, ready to move on to the next task. The key sequence I had to type was "C-x t C-u 8 0 - RET C-x e C-u 8 0 -", or 13 key presses.

There's simply no way you could do this reliably in 3 seconds with the mouse, since you'd have to make 2 complete round-trips to the mouse, to grab the scroll button (aka "thumb" or "elevator") and drag it to the top or bottom, then return to the keyboard to type out the text. A quick trial took me close to 15 seconds. Scrolling to the top does NOT move the cursor to the first character, so you also have to carefully position the cursor there.

Sure, you could practice it a bit, and maybe get it down to 10 seconds, but why bother? If you're going to do that exact operation frequently, you should write a macro for it.

Mouse use case #1 of 1: region selections

There are clearly some operations that will be faster with the mouse. Interacting with your window system outside Emacs is usually faster with the mouse, as is interacting with other applications that don't have Emacs-like keyboard navigation. But inside Emacs, I can only think of one thing that's faster with the mouse: region selection, particularly if you're selecting a rectangle.

Sometimes region selection with the keyboard is faster: for instance, setting the mark and holding down Ctrl-n to start selecting lines, or Ctrl-f to grow the selection by a character at a time. But the keyboard repeat rate, which is set in hardware, can feel annoyingly slow. I can see about 100 lines of text in a buffer window on my display, and selecting those lines with the keyboard (assuming I'm starting with the point somewhere in the middle) takes about 5 seconds. Selecting them with the mouse and returning to home row takes about 4 seconds. So when I just need to select lines in the visible area, the mouse usually isn't worth it.

However, if I need to select a region that's taller than my window size (by drag-selecting), or I need to select a region whose beginning and end columns both fall mid-line somewhere, then the mouse is the most reliably fast approach, and I'll use it happily.

Using the mouse for selections isn't turning off the UI, though, so it's only slightly related to this tip. The point is that I actually do timing experiments like this once in a while. My opinions about the GUI in Emacs are backed by 20 years of this kind of experimentation. And I'm recommending that you turn off the menus. Really.

Menus: lose 'em!

Menus are fine for exploration, for learning more about Emacs's capabilities. Unfortunately they can easily lull you into thinking the cover everything Emacs can do. However, many Emacs packages don't have any menu support -- it's only the super-meticulous package designer who goes to the effort of adding menu support. So if you're using the menus for browsing and exploring Emacs, you're missing out on a lot of functionality.

Another problem with menus is related to my rant about dialogs earlier: they don't scale well. If you want to provide the user with 1500 choices, putting them in a menu will tax the windowing system to the limits of its ability. Doing it in an Emacs buffer is trivial, and gives you a lot more real-estate in which to do nice layouts and groupings of the choices. Type M-x list-colors-display or M-x list-faces-display to see some examples of what I mean.

Another (huge) problem with menus is that they're not searchable, and they don't do auto-completion. You can easily find yourself digging way down some submenu heirarchy, thinking you're getting close, but you're actually barking down the wrong tree. They're nowhere near as flexible an exploration mechanism as searchable help -- this is every bit as true in Microsoft applications as it is in Emacs.

And finally, once you've memorized a menu action, you always have to go back to the menu (and possibly submenus) to perform it. The more often you do the action, the more time you've wasted compared to using a keyboard shortcut.

So I think Emacs menus are no good. They don't show you everything Emacs can do; they don't suggest alternatives when you can't find what you want; they're not capable of scaling to thousands of choices (so they're not a very good general-purpose UI mechanism, compared to something like a tree view), and they're slow to access even when you know how to use them.

In short: turn off those menus! And as for big shiny buttons, well, gosh. Anything important enough for a button is important enough for a fast keyboard shortcut.

Item 8: Learn the most important help functions

To find out all the keyboard shortcuts that are defined for the current buffer, type M-x describe-bindings. It shows you a list of keys and the commands that they're mapped to.

If you move the cursor to one of the commands in this list, and hit Enter, it'll show you the Help for that command.

To find out what a particular key does, use M-x describe-key, and then type the key sequence you're interested in. It goes directly to the Help for that key, if the key is currently bound to a command.

If you're interested in finding a command, and you have a guess as to the name but you aren't sure exactly what it's called, use M-x apropos, followed by a regular expression (see Item 9) to use for matching command names. All Emacs commands (as well as functions variables, and property lists) are kept in global tables that can be searched with M-x apropos.

If, for instance, you're looking for a function that sends a buffer all the way to the back of the list, you could type M-x apropos-command buffer, which shows about 200 available commands with the word "buffer" in the command name. Near the top is a command, bury-buffer, whose documentation says:

bury-buffer   M-x bury-buffer RET Command: Put BUFFER at the end of the list of all buffers.

Et voila. Just the command you were looking for. You can easily winnow the search through large lists by specifying a more restrictive regexp.

Perhaps the most important Help function is M-x info, which brings up Emacs's interactive, menu-driven Info engine. You should learn to use Info. It has thousands of pages of documentation, and it's hyperlinked (in its own custom way that predates the Web, unfortunately), so it's much easier to navigate than, say, man pages. Once you've mastered the navigation keys in Info, it's faster than navigating HTML help with a browser, even for local static files, in part because of Info's ability to perform searches across multiple info files, and in part because Emacs simply has better navigation capabilities than any web browser.

Item 9: Master Emacs's regular expressions

The best way to do this is to get yourself the Friedl Book, Mastering Regular Expressions. It's worth it. Every programmer should have a copy, no matter what language or editor you're using.

Emacs's regular expressions have some idiosyncracies that everyone dislikes, but they're not insurmountable, and once you've learned them, it opens up new horizons in editing power.

Two important regexp-related commands are isearch-forward-regexp and isearch-backward-regexp. These are by default bound to ESC C-r and ESC C-s, respectively. Those keys are lame, though. Any sequence that requires hitting the actual Escape key is lame, and Alt-Ctrl-s on my Compaq machine is invisible to Emacs; it brings up a system diagnostics dialog.

I have the isearch-*-regexp commands bound to Alt-r and Alt-s, since I use them so much. Alt-s isn't normally bound. The default Emacs binding for Alt-r is the command move-to-window-line, which you won't need, because you'll use Item 4 for moving around within the window.

Some modes insist on re-binding Alt-r and Alt-s, which is annoying, and I have a bunch of per-mode hacks to re-bind them, but I don't have all modes covered. If someone can suggest a way to bind Alt-r and Alt-s in such a way that they can't be overridden by any modes, please let me know -- I'd be muchly appreciative.

The next two important regexp-related commands are replace-regexp and query-replace-regexp. They function identically, prompting for a regular expression and a replacement string, but query-replace-regexp prompts you to type y or n for each possible replacement.

I use query-replace-regexp so frequently that I have an alias for it:

(defalias 'qrr 'query-replace-regexp)

That way I can type M-x qrr to invoke the function.

Other useful commands that take regexps are M-x list-matching-lines, which shows you all the lines in a buffer that match some regexp, and M-x apropos, which shows you all commands whose names match a given regexp.

The most Frequently Asked Question about Emacs regexps is: "How do I insert a newline into a regexp or the replacement string?" Hitting the Enter key simply tells the command that you're done entering the regexp, and it starts doing the replacements. (This is a very good reason for preferring query-replace-regexp over replace-regexp until you're 100% confident that your regexps are right on the first try. I'm still not there yet.)

The answer is that you need to insert a ^J character, which Emacs uses to represent newlines in functions and commands. At the point in the regexp or replacement where you need to insert a newline, hit Ctrl-q followed by Ctrl-j. Ctrl-q is Emacs's "quote" command: rather than executing the following keystroke, Emacs will insert the key into the current buffer or the minibuffer.

Some other useful things to know about Emacs regular expressions:

You need to double-escape ("\\") regexp metacharacters in strings in elisp code, but you single-escape them when entering regexps at the minibuffer.
Emacs code does so much matching of parens that Emacs regexps reverse the paren character and metacharacter. In an Emacs regexp, "(" and ")" match actual parens, while "$" and "$" create matching groups.
In the replacement string, use \1, \2, etc. for inserting match-groups from the regexp.
If you enter in a regexp and it doesn't work, undo the result if the command actually changed anything. Then type the command again, and when it prompts you, use the arrow keys to scroll up and down to find your regexp and replacement string, and modify them in place. This can save you a lot of typing and frustration.
You can yank text into the middle of a regexp you're typing out with most regexp commands, but NOT in the isearch-*-regexp commands. Yank does weird things in those. I'd love to know how to fix this.

Mastering regular expressions and the commands that use them is one of the most important components of becoming an Emacs Wizard.

Item 10: Master the fine-grained text manipulation commands

Emacs provides many small but useful commands for doing surgery on text. They add up to laserlike efficiency.

For starters, don't be tempted to re-bind Ctrl-k to a function that kills the whole line including the newline. I know that's the way kill-line works in all other editors. But it's clumsy and coarse compared to the way kill-line works by default. The default behavior, which kills the text to the end of the line but doesn't kill the newline, gives you finer-grained control, and leads to more efficient text surgery on the whole. Trust me: all Emacs users use other applications that only support the fat-fingered kill-whole-line version, so they've had plenty of opportunity to use both approaches. There's a reason the default is the way it is.

Keyboard Macros

I believe I can state without the slightest hint of exaggeration that Emacs keyboard macros are the coolest thing in the entire universe. They are, in a sense, as fine-grained and special-purpose as it gets, because you create them on the fly to solve specific editing problems. Whenever you find you need to make a specific, patterned change more than, say, 10 to 15 times, create a keyboard macro. It's really easy to do.

First it helps to do a trial run on the macro. Once you've done that, put your cursor at at the beginning of the first place to change, and use Ctrl-x ( to start recording the macro. Make your edits, and make sure to put the cursor in the corresponding place on the next line (or several lines down, as appropriate), so the macro will execute exactly the same pattern every time. To stop recording, type Ctrl-x ), and to invoke the macro, use Ctrl-x e (call-last-kbd-macro).

It's something of an art to define a robust macro -- you learn to use anchors like beginning-of-line and end-of line to make sure the macro is in a stable point before adding another action to it.

And incremental-search is useful for skipping forward to the next place to invoke the macro: if you use isearch within the macro, to find the place to start, then each invocation of the macro later will automatically perform the search each time. Very convenient.

And it's OK to make minor navigational mistakes while recording the macro. Just move the cursor back to where it should be and keep recording. The mistake will be replayed every time you execute the macro, but it'll happen so fast you'll be unlikely to notice.

The trick to getting good at macros is to be persistent: make sure you get the macro working, and don't give up, even if it means spending more time fiddling with the macro than making the edits manually. The next time around it'll be easier, and eventually they'll become second nature. Keyboard macros are one of the most powerful features of Emacs, and they can make your life much easier.

One last tip for keyboard macros: typically you'll play them dozens or even hundreds of times in a file, or across multiple files. You should bind call-last-kbd-macro to a single keystroke, such as the F5 key. You'll usually be "babysitting" the job, pressing the key over and over, watching the changes as you make them to ensure you don't hork something unintentionally. So it's fine to have it bound to a "faraway" key like F5. You can set it up like so:

(global-set-key [f5] 'call-last-kbd-macro)

Transpose-* functions

Finally you'll find the transpose-* commands surprisingly useful once you get used to them, even though they look like gimmicks. Probably the most useful is transpose-words. You'd be amazed at how often you find yourself using this command, which is bound to Alt-t. It has two uses: swapping two adjacent words, and dragging a single word forward or backward in a sentence.

The transpose-words function is aware of mode-specific syntax and word boundaries, so, for instance, putting the cursor between these two words:

([this])-is

and hitting Alt-t will result in:

([is])-this

You can transpose words across hyphens, HTML tags, and basically all punctuation. It's a pretty handy feature when you need it.

When you transpose 2 words, e.g. "dog food", the leftmost word swaps with the rightmost (assuming the cursor is somewhere between the beginning of the first word and the end of the second one), resulting in "food dog". But the cursor moves so that it's still to the right of the word that moved to the right. So if you apply it repeatedly, the word moves along the sentence to the right. I don't think there's a built-in way to make a "drag-word-left", but it would be easy to write a short function to do it. The corresponding Eclipse plug-in would be 5,000 lines of code in 60 source files and would take nine days to write and debug.

You can also transpose chars, lines, sentences, and paragraphs. These are all useful as you perform editing operations on your raw text and even your source code. Experiment with them and try to remember that they're there, and eventually they'll also become second-nature.

Tune in next time...

At some point I'll have collected and documented 50 tips, at which point Scott Meyers can just eat his heart out. I made a valiant effort to do them all in one sitting, but I gave up just now and changed this blog's heading from "50 Specific Ways..." to "10 Specific Ways..." At least I did it this entry all in one sitting of a little over 2 hours. An Eclipse user would have spent that much time digging through the manuals, looking for a refactoring tool that can write blogs. Fat chance.

For upcoming tips, a few that come to mind as being particularly useful are:

fill-paragraph (Alt-p) -- intelligently line-wraps your text for you: an absolute must, and it works inside source-code comments.
gnuserv: automatically open certain document types (including View Source in your browser) in Emacs.
M-x Dired: a powerful way to manage your files and directories. It can do things that NO other tool can do (at least that I'm aware of), including renaming arbitrary groups of files from one regexp to another.
whitespace-manipulation commands: C-x C-o (delete-blank-lines), delete-trailing-whitespace, tabify and untabify, indent-region, and so on.
nxml-mode: the ONLY way to fly when you're editing XML. Authored by XML guru James Clark; it knocks the socks off any IDE-based XML editor out there today.
picture-mode: the best way to draw ascii art, and useful in a surprising number of situations.
minibuffer management: mastering recursive edits, learning how to abort from various situations, command completions, and other command-entry trickery.
effortless navigation: re-bind a few keys so that you can move the cursor in any direction, by chars or words, by holding down Alt and just pressing various letter keys.
region management: choosing a non-disgusting color for the highlighted region, covering region-related commands.
rectangle commands: yet another incredibly important set of related commands with no analogues in other editors. Once again, you'll wonder how you lived without them.
emacs shells: tips and tricks for getting the most out of a bash command shell running as an Emacs subprocess.
align-regexp: my new favorite command. just learned it recently, and I use it almost every day.
frame initialization: put Emacs exactly where you want it, every time it starts up, by auto-detecting the screen dimensions and computing where it should be.
using the goal column: things every Emacs power-user should know.
setting the fill column: how to get the most out of fill-region and fill-paragraph.
optimizing OS settings, such as speeding up your keyboard repeat rate, choosing an ideal Emacs font, and so on.
browsing and editing archives: tar, gzip, zip, jar, etc. Most people have no idea this feature exists, and it's nothing short of amazing.
advanced keybinding: learn the syntax for binding function keys, home/end, and other oddball keys. Learn how to make keybindings local to a particular buffer or mode.
mastering the kill ring, including using Derek Upham's fancy new mode for viewing its contents.
mastering Info: customizing your Info dir, finding and adding in new Info files, and advanced navigation and bookmarking.
using M-x customize: learn how this beast works, and how to use it or avoid it as needed.
utility apps: M-x calendar, M-x calc, and others.

The list goes on and on... ah, well, I'll get around to them someday.

And with that, it's a wrap! I'm heading to bed.

published Jan 23, 2005
last update, Mar 12, 2006

Comments

Some random comments:

To yank at the i-search prompt, use M-y instead of C-y. The emacs info node on Incremental Search talks about the rebinding of C-y, C-w, etc at the i-search prompt.

To repeat execution of the last kbd macro, after hitting C-xe to run it once, keep hitting just the 'e' key for repeated execution.

Zap-to-char (M-z) is incredibly useful if you need to change myDatafileHandler to myStreamHandler and point is at D (the first char to change). Simple M-z e to zap the "Datafile" part and type in the replacement. This is not orthogonal to the backward-kill-word (bound to C-backspace for me since that works in windows browser windows as well as everything else) so there's a feel needed for which is optimal when, which for me mostly depends on where point is already.

Saturday, October 27, 2007

Newbie: Howto Improve Your Productivity With Emacs

Effective Emacs