Mass renaming with linux shell

When you do mv *.aaa *.bbb, it’s not obvious to a newbie that it won’t work.

The solution is a bit more tricky and needs piping (|) and sed.

23 September 2003, by Mortimer

1Renaming

So, if you wanted to rename all the .php3 to .php, this would be the command:

ls -d *.php3 | sed 's/$.*$.php3$/mv "&" "\1.php"/' | sh

so what does this do?

ls -d *.php3 outputs a list of all the php3 files in the directory. This list is piped to the second command,
sed ’s/$.*$.php3$/mv "&" "\1.php"/’ is not as obvious for newbies. sed is a tool to edit stream. Here the stream is the list of files. the argument passed to sed ’s/$.*$.html$/mv "&" "\1.php"/’ is what should be edited:
s tells sed to replace a string by another,
(.*\).php3$ is a regular expression to tell that the string we will want to replace is anyname.php3,
mv "&" "\1.php" is the replacement string. the \1 is there to take the name of the file in the first string and put it there. & refers to the portion of the pattern space which matched,

| sh takes the output of sed and executes it (sh is the shell interpreter). hence executing the list of mv anyname.php3 anyname.php.

2Anything

In fact this trick could be used for anything. The sole part that will change would be the replacement string in the sed command.

Then if you want to replace all the php3 references in a file by php you’ll use:

sed 's/\.php3/.php/g'

so for a mass replacement, the command would be:

ls -d *.php3 | sed 's/$.*$.php3$/sed "s\/\\.php3\/.php\/g" "&" > \1.php' | sh

remark that you have to escape all the / and \. And that in the same time we will rename the files to php :).

Date of online publication: 23 September 2003
last-update: 11 May 2004

All the versions of this article:

English
français

Forum messages 7
visits:

25924

Reply to this article

7 Forum messages

> Mass renaming with linux shell
24 December 2003 10:04, by tendays

I prefer:

for x in `ls -d *.php3 | sed ’s/.php3$//’ `; do mv $x.php3 $x.php; done

It is more readable (the sed will just strip the extension away and then I put it again in the mv command - just have to be careful not mistaking quotes)

(By the way the . in the regexp for sed will stand for any character but we know it will always be a point because of the regexp for ls)

Tendays

Reply to this message
> Mass renaming with linux shell
1 January 2004 12:17

While using sed is a more generalized way for doing file renaming, in the case of changing the extension, basename makes for a simpler example:

for i in * ; do mv $i `basename $i .php3`.php ; done

Reply to this message
- > Mass renaming with linux shell 1 January 2004 12:21
  
  Oops - mind the quoting. A safer example:
  
  for i in * ; do mv "$i" `basename "$i" .php3`.php ; done
  
  Reply to this message
Mass renaming with linux shell
15 December 2005 23:25, by Nathan
Greetings to all.

In the last exemple a “/” is missing at the end of the pattern. To work it should read:
ls -d *.php3 | sed 's/$.*$.php3$/sed "s\/\\.php3\/.php\/g" "&" > \1.php/' | sh

By the way, you forgot to explain the “-d” you use in the “ls” command.

There is a rename command under Linux that does the same job:
rename .php3 .php *.php3

But
1. it does only that and
2. it does not teach much about scripting, does it?
Indeed with the same technique you could manipulate the filename much more:
ls -d ecrire/inc_charsets.php3 | sed 's/$.*$.php3$/sed "s\/\\.php3\/.php\/g" "&" > \1.php/;s/> $.*$some$.*$/> \1other\2/' | sh
Same suffix replacement as before (php3 to php) but also replaces the first occurrence of string “some” by string “other” in the basename.

And now a slightly trickier one:
ls -d *.php3 | sed 's/[.]php3$//; h; s/.*/sed "s\/\\.php3\/.php\/g" "&.php3" > /; x; y/abcde/12345/; s/$/.php/; H; x; s/\n/ /' | sh
Same as above (php3 suffix to php) and now replaces all occurrences of letters a,b,c,d and e by their matching figure. It has much more possible uses as you can see.

I let you explain how it works...

Ok, ok: I’ll give it a try...

It uses the "Hold space" of sed which comes very handy (see sed manual for h, H, x, and y commands):
- s/[.]php3$//: removes the “.php3” suffix
  (The brackets around the “.” makes sure it is a dot we are replacing. It is probably useless because of the ls command but I like to be on the safe side )
- h: saves the original basename in the hold space
- s/.*/sed "s\/\\.php3\/.php\/g" "&.php3" > /: prepares the command that will replace .php3 by .php inside the file
- x: exchanges hold and current spaces (sends the prepared command to hold space and get the original basename back in current space)
- y/abcde/12345/: replaces all character in the first member by their corresponding character in second member
- s/$/.php/: appends the suffix to the new name
- H: appends the current space to the hold space inserting a newline in between
- x: exchange hold and current spaces
- s/\n/ /: replaces the newline by a space
Keep up the good work,
Nate

Reply to this message
Mass renaming with linux shell
24 June 2007 23:50, by brian

Great tip. I did a google search for "linux mass file rename" and your page was among the top. A simple and effective solution. Thanks!
Reply to this message
- Mass renaming with linux shell 20 September 2007 18:38
  
  Same here...just did the Google search and this page came out on top....thanks for the help.
  Reply to this message
Mass renaming with linux shell
4 October 2007 01:17, by Bodhi.zazen

You can also use the rename command.

For example, to replace spaces in file names with _ :

rename -v ’s/\ /\_/g’ *

Reply to this message

6V8 - Production of my Mind