Mdthbs

A good replacement Linux tool is rpl, that was originally written for the Debian project, so it is available with apt-get install rpl in any Debian derived distro, and may be for others, but otherwise you can download the tar.gz file in SourgeForge.

Simplest example of use:

 $ rpl old_string new_string test.txt

Note that if the string contain spaces it should be enclosed in quotation marks. By default rpl take care of capital letters but not of complete words, but you can change these defaults with options -i (ignore case) and -w (whole words). You can also specify multiple files:

 $ rpl -i -w "old string" "new string" test.txt test2.txt

Or even specify the extensions (-x) to search or even search recursively (-R) in the directory:

 $ rpl -x .html -x .txt -R old_string new_string test*

You can also search/replace in interactive mode with -p (prompt) option:

The output show the numbers of files/string replaced and the type of search (case in/sensitive, whole/partial words), but it can be silent with the -q (quiet mode) option, or even more verbose, listing line numbers that contain matches of each file and directory with -v (verbose mode) option.

Other options that are worth remembering are -e (honor escapes) that allow regular expressions, so you can search also tabs (t), new lines (n),etc. Even you can use -f to force permissions (of course, only when the user have write permissions) and -d to preserve the modification times`).

Finally, if you are unsure of which will make exactly, use the -s (simulate mode).

How to do a search and replace over multiple files suggests:

You could also use find and sed, but I find that this little line of
perl works nicely.

perl -pi -w -e 's/search/replace/g;' *.php

• -e means execute the following line of code.


• -i means edit in-place


• -w write warnings


• -p loop over the input file, printing each line after the script is applied to it.

My best results come from using perl and grep (to ensure that file have the search expression )

perl -pi -w -e 's/search/replace/g;' $( grep -rl 'search' )

You can use Vim in Ex mode:

replace string ALF with BRA in all files in the current directory?

for CHA in *

do

  ex -sc '%s/ALF/BRA/g' -cx "$CHA"

done

do the same recursively for sub directories?

find -type f -exec ex -sc '%s/ALF/BRA/g' -cx {} ';'

replace only if the file name matches another string?

for CHA in *.txt

do

  ex -sc '%s/ALF/BRA/g' -cx "$CHA"

done

replace only if the string is found in a certain context?

ex -sc 'g/DEL/s/ALF/BRA/g' -cx file

replace if the string is on a certain line number?

ex -sc '2s/ALF/BRA/g' -cx file

replace multiple strings with the same replacement

ex -sc '%s/vALF|ECH/BRA/g' -cx file

replace multiple strings with different replacements

ex -sc '%s/ALF/BRA/g|%s/FOX/GOL/g' -cx file

I used this:

grep -r "old_string" -l | tr 'n' ' ' | xargs sed -i 's/old_string/new_string/g'

1. List all files that contain old_string.




2. Replace newline in result with spaces (so that the list of files can be fed to sed.




3. Run sed on those files to replace old string with new.

Update: The above result will fail on filenames that contain whitespaces. Instead, use:

grep --null -lr "old_string" | xargs --null sed -i 's/old_string/new_string/g'

From a user's perspective, a nice & simple Unix tool that does the job perfectly is qsubst. For example,

% qsubst foo bar *.c *.h

will replace foo with bar in all my C files. A nice feature is that qsubst will do a query-replace, i.e., it will show me each occurrence of foo and ask whether I want to replace it or not. [You can replace unconditionally (no asking) with -go option, and there are other options, e.g., -w if you only want to replace foo when it is a whole word.]

How to get it: qsubst was invented by der Mouse (from McGill) and posted to comp.unix.sources 11(7) in Aug. 1987. Updated versions exist. For example, the NetBSD version qsubst.c,v 1.8 2004/11/01 compiles and runs perfectly on my mac.

I needed something that would provide a dry-run option and would work recursively with a glob, and after trying to do it with awk and sed I gave up and instead did it in python.

The script searches recursively all files matching a glob pattern (e.g. --glob="*.html") for a regex and replaces with the replacement regex:

find_replace.py [--dir=my_folder] 

    --search-regex=<search_regex> 

    --replace-regex=<replace_regex> 

    --glob=[glob_pattern] 

    --dry-run

^{Every long option such as --search-regex has a corresponding short option, i.e. -s. Run with -h to see all options.}

For example, this will flip all dates from 2017-12-31 to 31-12-2017:

python replace.py --glob=myfile.txt 

    --search-regex="(d{4})-(d{2})-(d{2})" 

    --replace-regex="3-2-1" 

    --dry-run --verbose

import os

import fnmatch

import sys

import shutil

import re



import argparse



def find_replace(cfg):

    search_pattern = re.compile(cfg.search_regex)



    if cfg.dry_run:

        print('THIS IS A DRY RUN -- NO FILES WILL BE CHANGED!')



    for path, dirs, files in os.walk(os.path.abspath(cfg.dir)):

        for filename in fnmatch.filter(files, cfg.glob):



            if cfg.print_parent_folder:

                pardir = os.path.normpath(os.path.join(path, '..'))

                pardir = os.path.split(pardir)[-1]

                print('[%s]' % pardir)

            filepath = os.path.join(path, filename)



            # backup original file

            if cfg.create_backup:

                backup_path = filepath + '.bak'



                while os.path.exists(backup_path):

                    backup_path += '.bak'

                print('DBG: creating backup', backup_path)

                shutil.copyfile(filepath, backup_path)



            with open(filepath) as f:

                old_text = f.read()



            all_matches = search_pattern.findall(old_text)



            if all_matches:



                print('Found {} matches in file {}'.format(len(all_matches), filename))



                new_text = search_pattern.sub(cfg.replace_regex, old_text)



                if not cfg.dry_run:

                    with open(filepath, "w") as f:

                        print('DBG: replacing in file', filepath)

                        f.write(new_text)

                else:

                    for idx, matches in enumerate(all_matches):

                        print("Match #{}: {}".format(idx, matches))



                    print("NEW TEXT:n{}".format(new_text))



            elif cfg.verbose:

                print('File {} does not contain search regex "{}"'.format(filename, cfg.search_regex))





if __name__ == '__main__':



    parser = argparse.ArgumentParser(description='''DESCRIPTION:

    Find and replace recursively from the given folder using regular expressions''',

                                     formatter_class=argparse.RawDescriptionHelpFormatter,

                                     epilog='''USAGE:

    {0} -d [my_folder] -s <search_regex> -r <replace_regex> -g [glob_pattern]



    '''.format(os.path.basename(sys.argv[0])))



    parser.add_argument('--dir', '-d',

                        help='folder to search in; by default current folder',

                        default='.')



    parser.add_argument('--search-regex', '-s',

                        help='search regex',

                        required=True)



    parser.add_argument('--replace-regex', '-r',

                        help='replacement regex',

                        required=True)



    parser.add_argument('--glob', '-g',

                        help='glob pattern, i.e. *.html',

                        default="*.*")



    parser.add_argument('--dry-run', '-dr',

                        action='store_true',

                        help="don't replace anything just show what is going to be done",

                        default=False)



    parser.add_argument('--create-backup', '-b',

                        action='store_true',

                        help='Create backup files',

                        default=False)



    parser.add_argument('--verbose', '-v',

                        action='store_true',

                        help="Show files which don't match the search regex",

                        default=False)



    parser.add_argument('--print-parent-folder', '-p',

                        action='store_true',

                        help="Show the parent info for debug",

                        default=False)



    config = parser.parse_args(sys.argv[1:])



    find_replace(config)

^{Here is an updated version of the script which highlights the search terms and replacements with different colors.}

ripgrep (command name rg) is a grep tool, but supports search and replace as well.

$ cat ip.txt

dark blue and light blue

light orange

blue sky

$ # by default, line number is displayed if output destination is stdout

$ # by default, only lines that matched the given pattern is displayed

$ # 'blue' is search pattern and -r 'red' is replacement string

$ rg 'blue' -r 'red' ip.txt

1:dark red and light red

3:red sky



$ # --passthru option is useful to print all lines, whether or not it matched

$ # -N will disable line number prefix

$ # this command is similar to: sed 's/blue/red/g' ip.txt

$ rg --passthru -N 'blue' -r 'red' ip.txt

dark red and light red

light orange

red sky

rg doesn't support in-place option, so you'll have to do it yourself

$ # -N isn't needed here as output destination is a file

$ rg --passthru 'blue' -r 'red' ip.txt > tmp.txt && mv tmp.txt ip.txt

$ cat ip.txt

dark red and light red

light orange

red sky

See Rust regex documentation for regular expression syntax and features. The -P switch will enable PCRE2 flavor. rg supports Unicode by default.

$ # non-greedy quantifier is supported

$ echo 'food land bark sand band cue combat' | rg 'foo.*?ba' -r 'X'

Xrk sand band cue combat



$ # unicode support

$ echo 'fox:αλεπού,eagle:αετός' | rg 'p{L}+' -r '($0)'

(fox):(αλεπού),(eagle):(αετός)



$ # set operator example, remove all punctuation characters except . ! and ?

$ para='"Hi", there! How *are* you? All fine here.'

$ echo "$para" | rg '[[:punct:]--[.!?]]+' -r ''

Hi there! How are you? All fine here.



$ # use -P if you need even more advanced features

$ echo 'car bat cod map' | rg -P '(bat|map)(*SKIP)(*F)|w+' -r '[$0]'

[car] bat [cod] map

Like grep, the -F option will allow fixed strings to be matched, a handy option which I feel sed should implement too.

$ printf '2.3/[4]*6nfoon5.3-[4]*9n' | rg --passthru -F '[4]*' -r '2'

2.3/26

foo

5.3-29

Another handy option is -U which enables multiline matching

$ # (?s) flag will allow . to match newline characters as well

$ printf '42nHi therenHave a Nice Day' | rg --passthru -U '(?s)the.*ice' -r ''

42

Hi  Day

rg can handle dos-style files too

$ # same as: sed -E 's/w+(r?)$/1231/'

$ printf 'hi thererngood dayrn' | rg --passthru --crlf 'w+$' -r '123'

hi 123

good 123

Another advantage of rg is that it is likely to be faster than sed

$ # for small files, initial processing time of rg is a large component

$ time echo 'aba' | sed 's/a/b/g' > f1

real    0m0.002s

$ time echo 'aba' | rg --passthru 'a' -r 'b' > f2

real    0m0.007s



$ # for larger files, rg is likely to be faster

$ # 6.2M sample ASCII file

$ wget https://norvig.com/big.txt    

$ time LC_ALL=C sed 's/bcatb/dog/g' big.txt > f1

real    0m0.060s

$ time rg --passthru 'bcatb' -r 'dog' big.txt > f2

real    0m0.048s

$ diff -s f1 f2

Files f1 and f2 are identical



$ time LC_ALL=C sed -E 's/b(w+)(s+1)+b/1/g' big.txt > f1

real    0m0.725s

$ time rg --no-pcre2-unicode --passthru -wP '(w+)(s+1)+' -r '$1' big.txt > f2

real    0m0.093s

$ diff -s f1 f2

Files f1 and f2 are identical