Mass renaming with linux shell

When you do mv *.aaa *.bbb, it’s not obvious to a newbie that it won’t work.

The solution is a bit more tricky and needs piping (|) and sed.

23 September 2003, by Mortimer

1Renaming

So, if you wanted to rename all the .php3 to .php, this would be the command:

ls -d *.php3 | sed 's/$.*$.php3$/mv "&" "\1.php"/' | sh

so what does this do?

ls -d *.php3 outputs a list of all the php3 files in the directory. This list is piped to the second command,
sed ’s/$.*$.php3$/mv "&" "\1.php"/’ is not as obvious for newbies. sed is a tool to edit stream. Here the stream is the list of files. the argument passed to sed ’s/$.*$.html$/mv "&" "\1.php"/’ is what should be edited:
s tells sed to replace a string by another,
(.*\).php3$ is a regular expression to tell that the string we will want to replace is anyname.php3,
mv "&" "\1.php" is the replacement string. the \1 is there to take the name of the file in the first string and put it there. & refers to the portion of the pattern space which matched,

| sh takes the output of sed and executes it (sh is the shell interpreter). hence executing the list of mv anyname.php3 anyname.php.

2Anything

In fact this trick could be used for anything. The sole part that will change would be the replacement string in the sed command.

Then if you want to replace all the php3 references in a file by php you’ll use:

sed 's/\.php3/.php/g'

so for a mass replacement, the command would be:

ls -d *.php3 | sed 's/$.*$.php3$/sed "s\/\\.php3\/.php\/g" "&" > \1.php' | sh

remark that you have to escape all the / and \. And that in the same time we will rename the files to php :).

Date of online publication: 23 September 2003
last-update: 11 May 2004

All the versions of this article:

English
français

Forum messages 13
visits:

69446

13 Forum messages

> Mass renaming with linux shell
24 December 2003 10:04, by tendays

I prefer:

for x in `ls -d *.php3 | sed ’s/.php3$//’ `; do mv $x.php3 $x.php; done

It is more readable (the sed will just strip the extension away and then I put it again in the mv command - just have to be careful not mistaking quotes)

(By the way the . in the regexp for sed will stand for any character but we know it will always be a point because of the regexp for ls)

Tendays
> Mass renaming with linux shell
1 January 2004 12:17

While using sed is a more generalized way for doing file renaming, in the case of changing the extension, basename makes for a simpler example:

for i in * ; do mv $i `basename $i .php3`.php ; done
- > Mass renaming with linux shell 1 January 2004 12:21
  
  Oops - mind the quoting. A safer example:
  
  for i in * ; do mv "$i" `basename "$i" .php3`.php ; done
Mass renaming with linux shell
15 December 2005 23:25, by Nathan
Greetings to all.

In the last exemple a “/” is missing at the end of the pattern. To work it should read:
ls -d *.php3 | sed 's/$.*$.php3$/sed "s\/\\.php3\/.php\/g" "&" > \1.php/' | sh

By the way, you forgot to explain the “-d” you use in the “ls” command.

There is a rename command under Linux that does the same job:
rename .php3 .php *.php3

But
1. it does only that and
2. it does not teach much about scripting, does it?
Indeed with the same technique you could manipulate the filename much more:
ls -d ecrire/inc_charsets.php3 | sed 's/$.*$.php3$/sed "s\/\\.php3\/.php\/g" "&" > \1.php/;s/> $.*$some$.*$/> \1other\2/' | sh
Same suffix replacement as before (php3 to php) but also replaces the first occurrence of string “some” by string “other” in the basename.

And now a slightly trickier one:
ls -d *.php3 | sed 's/[.]php3$//; h; s/.*/sed "s\/\\.php3\/.php\/g" "&.php3" > /; x; y/abcde/12345/; s/$/.php/; H; x; s/\n/ /' | sh
Same as above (php3 suffix to php) and now replaces all occurrences of letters a,b,c,d and e by their matching figure. It has much more possible uses as you can see.

I let you explain how it works...

Ok, ok: I’ll give it a try...

It uses the "Hold space" of sed which comes very handy (see sed manual for h, H, x, and y commands):
- s/[.]php3$//: removes the “.php3” suffix
  (The brackets around the “.” makes sure it is a dot we are replacing. It is probably useless because of the ls command but I like to be on the safe side )
- h: saves the original basename in the hold space
- s/.*/sed "s\/\\.php3\/.php\/g" "&.php3" > /: prepares the command that will replace .php3 by .php inside the file
- x: exchanges hold and current spaces (sends the prepared command to hold space and get the original basename back in current space)
- y/abcde/12345/: replaces all character in the first member by their corresponding character in second member
- s/$/.php/: appends the suffix to the new name
- H: appends the current space to the hold space inserting a newline in between
- x: exchange hold and current spaces
- s/\n/ /: replaces the newline by a space
Keep up the good work,
Nate
Mass renaming with linux shell
24 June 2007 23:50, by brian

Great tip. I did a google search for "linux mass file rename" and your page was among the top. A simple and effective solution. Thanks!
- Mass renaming with linux shell 20 September 2007 18:38
  
  Same here...just did the Google search and this page came out on top....thanks for the help.
Mass renaming with linux shell
4 October 2007 01:17, by Bodhi.zazen

You can also use the rename command.

For example, to replace spaces in file names with _ :

rename -v ’s/\ /\_/g’ *
Mass renaming with linux shell
25 November 2008 01:46

Great tip. Saved me a lot of time.

Explaining every part of the command was a good thing also.
Mass renaming with linux shell
14 January 2009 23:53, by NronKsr

This will do the same thing and is a whole lot simpler. command is: "rename from to files"

rename *.php3 *.php *.php

- NronKsr
- Mass renaming with linux shell 3 February 2009 10:26, by Mortimer
  
  yes, rename is available on some systems, but not everywhere though
Mass renaming with linux shell
6 April 2009 23:00, by warzauwynn

This is a great write-up on the logic behind doing such a task. For those who want a tool that is portable and exists on many modern linux distributions, check out prename. It’s written in perl so it can be copied to systems that don’t include it by default. It has very simple syntax: prename ’s/\.php3$/\.php/’ *.php3

http://man.he.net/man1/prename
Mass renaming with linux shell
19 December 2009 21:55, by James Waples

Thanks very much - I renamed a bunch of *.php to *.html using this script
- Mass renaming with linux shell 4 February 00:12, by Sir_Ty
  
  BTW if you get a "Unrecognized character \xE2" error, it’s probably the single quotes used in some of the examples, as they may come out as a different character if you copy and paste the command. Type it instead - you learn more that way too (I did!).
  See online : Example of the quoting problem

6V8 - Production of my Mind