Daddy, I found it!, 15 Awesome Linux Find Command Examples

A while back we reviewed 15 practical find command examples (Part I). Find command can do lot more than just searching for files based on name.

In this article (Part 2), let us discuss 15 advanced examples of find commandincluding — finding files based on the time it is accessed, modified or changed, finding files comparatively, performing operation on found files etc.,

Ramesh Natarajan: That is my sweet little daughter in that picture. She was very happy to spot the sea lion in the California Long Beach Aquarium.

Find Files Based on Access / Modification / Change Time

You can find files based on following three file time attribute.

  1. Access time of the file. Access time gets updated when the file accessed.
  2. Modification time of the file. Modification time gets updated when the file content modified.
  3. Change time of the file. Change time gets updated when the inode data changes.

In the following examples, the difference between the min option and the time option is the argument.

  • min argument treats its argument as minutes. For example, min 60 = 60 minutes (1 hour).
  • time argument treats its argument as 24 hours. For example, time 2 = 2*24 hours (2 days).
  • While doing the 24 hours calculation, the fractional parts are ignored so 25 hours is taken as 24 hours, and 47 hours is also taken as 24 hours, only 48 hours is taken as 48 hours. To get more clarity refer the -atime section of the find command man page.

Example 1: Find files whose content got updated within last 1 hour

To find the files based up on the content modification time, the option -mmin, and -mtime is used. Following is the definition of mmin and mtime from man page.

  • -mmin n File’s data was last modified n minutes ago.
  • -mtime n File’s data was last modified n*24 hours ago.

Following example will find files in the current directory and sub-directories, whose content got updated within last 1 hour (60 minutes)

# find . -mmin -60

In the same way, following example finds all the files (under root file system /) that got updated within the last 24 hours (1 day).

# find / -mtime -1

Example 2: Find files which got accessed before 1 hour

To find the files based up on the file access time, the option -amin, and -atime is used. Following is the definition of amin and atime from find man page.

  • -amin n File was last accessed n minutes ago
  • -atime n File was last accessed n*24 hours ago

Following example will find files in the current directory and sub-directories, which got accessed within last 1 hour (60 minutes)

# find -amin -60

In the same way, following example finds all the files (under root file system /) that got accessed within the last 24 hours (1 day).

# find / -atime -1

Example 3: Find files which got changed exactly before 1 hour

To find the files based up on the file inode change time, the option -cmin, and -ctime is used. Following is the definition of cmin and ctime from find man page.

  • -cmin n File’s status was last changed n minutes ago.
  • -ctime n File’s status was last changed n*24 hours ago.

Following example will find files in the current directory and sub-directories, which changed within last 1 hour (60 minutes)

# find . -cmin -60

In the same way, following example finds all the files (under root file system /) that got changed within the last 24 hours (1 day).

# find / -ctime -1

Example 4: Restricting the find output only to files. (Display only files as find command results)

The above find command’s will also show the directories because directories gets accessed when the file inside it gets accessed. But if you want only the files to be displayed then give -type f in the find command as

The following find command displays files that are accessed in the last 30 minutes.

# find /etc/sysconfig -amin -30
.
./console
./network-scripts
./i18n
./rhn
./rhn/clientCaps.d
./networking
./networking/profiles
./networking/profiles/default
./networking/profiles/default/resolv.conf
./networking/profiles/default/hosts
./networking/devices
./apm-scripts
[Note: The above output contains both files and directories]

# find /etc/sysconfig -amin -30 -type f
./i18n
./networking/profiles/default/resolv.conf
./networking/profiles/default/hosts
[Note: The above output contains only files]

Example 5: Restricting the search only to unhidden files. (Do not display hidden files in find output)

When we don’t want the hidden files to be listed in the find output, we can use the following regex.
The below find displays the files which are modified in the last 15 minutes. And it lists only the unhidden files. i.e hidden files that starts with a . (period) are not displayed in the find output.

# find . -mmin -15 \( ! -regex ".*/\..*" \)

Finding Files Comparatively Using Find Command

Human mind can remember things better by reference such as, i want to find files which i edited after editing the file “test”. You can find files by referring to the other files modification as like the following.

Example 6: Find files which are modified after modification of a particular FILE

Syntax: find -newer FILE

Following example displays all the files which are modified after the /etc/passwd files was modified. This is helpful, if you want to track all the activities you’ve done after adding a new user.

# find -newer /etc/passwd

Example 7: Find files which are accessed after modification of a specific FILE

Syntax: find -anewer FILE

Following example displays all the files which are accessed after modifying /etc/hosts. If you remember adding an entry to the /etc/hosts and would like to see all the files that you’ve accessed since then, use the following command.

# find -anewer /etc/hosts

Example 8: Find files whose status got changed after the modification of a specific FILE.

Syntax: find -cnewer FILE

Following example displays all the files whose status got changed after modifying the /etc/fstab. If you remember adding a mount point in the /etc/fstab and would like to know all the files who status got changed since then, use the following command.

find -cnewer /etc/fstab

Perform Any Operation on Files Found From Find Command

We have looked at many different ways of finding files using find command in this article and also in our previous article. If you are not familiar in finding files in different ways, i strongly recommend you to read the part 1.

This section explains about how to do different operation on the files from the find command. i.e how to manipulate the files returned by the find command output.

We can specify any operation on the files found from find command.

find <CONDITION to Find files> -exec <OPERATION> \;

The OPERATION can be anything such as:

  • rm command to remove the files found by find command.
  • mv command to rename the files found.
  • ls -l command to get details of the find command output files.
  • md5sum on find command output files
  • wc command to count the total number of words on find command output files.
  • Execute any Unix shell command on find command output files.
  • or Execute your own custom shell script / command on find command output files.

Example 9: ls -l in find command output. Long list the files which are edited within the last 1 hour.

# find -mmin -60
./cron
./secure

# find -mmin -60 -exec ls -l {} \;
-rw-------  1 root root 1028 Jun 21 15:01 ./cron
-rw-------  1 root root 831752 Jun 21 15:42 ./secure

Example 10: Searching Only in the Current Filesystem

System administrators would want to search in the root file system, but not in the other mounted partitions. When you have multiple partitions mounted, and if you want to search in /. You can do the following.

Following command will search for *.log files starting from /. i.e If you have multiple partitions mounted under / (root), the following command will search all those mounted partitions.

# find / -name "*.log"

This will search for the file only in the current file system. Following is the xdev definition from find man page:

  • -xdev Don’t descend directories on other filesystems.

Following command will search for *.log files starting from / (root) and only in the current file system. i.e If you have multiple partitions mounted under / (root), the following command will NOT search all those mounted partitions.

# find / -xdev -name "*.log"

Example 11: Using more than one { } in same command

Manual says only one instance of the {} is possible. But you can use more than one {} in the same command as shown below.

# find -name "*.txt" cp {} {}.bkup \;

Using this {} in the same command is possible but using it in different command it is not possible, say you want to rename the files as following, which will not give the expected result.

find -name "*.txt" -exec mv {} `basename {} .htm`.html \;

Example 12: Using { } in more than one instance.

You can simulate it by writing a shell script as shown below.

# mv "$1" "`basename "$1" .htm`.html"

These double quotes are to handle spaces in file name. And then call that shell script from thefind command as shown below.

find -name "*.html" -exec ./mv.sh '{}' \;

So for any reason if you want the same file name to be used more than once then writing the simple shell script and passing the file names as argument is the simplest way to do it.

Example 13: Redirecting errors to /dev/null

Redirecting the errors is not a good practice. An experienced user understands the importance of getting the error printed on terminal and fix it.

Particularly in find command redirecting the errors is not a good practice. But if you don’t want to see the errors and would like to redirect it to null do it as shown below.

find -name "*.txt" 2>>/dev/null

Sometimes this may be helpful. For example, if you are trying to find all the *.conf file under / (root) from your account, you may get lot of “Permission denied” error message as shown below.

$ find / -name "*.conf"
/sbin/generate-modprobe.conf
find: /tmp/orbit-root: Permission denied
find: /tmp/ssh-gccBMp5019: Permission denied
find: /tmp/keyring-5iqiGo: Permission denied
find: /var/log/httpd: Permission denied
find: /var/log/ppp: Permission denied
/boot/grub/grub.conf
find: /var/log/audit: Permission denied
find: /var/log/squid: Permission denied
find: /var/log/samba: Permission denied
find: /var/cache/alchemist/printconf.rpm/wm: Permission denied
[Note: There are two valid *.conf files burned in the "Permission denied" messages]

So, if you want to just view the real output of the find command and not the “Permission denied” error message you can redirect the error message to /dev/null as shown below.

$ find / -name "*.conf" 2>>/dev/null
/sbin/generate-modprobe.conf
/boot/grub/grub.conf
[Note: All the "Permission denied" messages are not displayed]

Example 14: Substitute space with underscore in the file name.

Audio files you download from internet mostly come with the spaces in it. But having space in the file name is not so good for Linux kind of systems. You can use the find and rename command combination as shown below to rename the files, by substituting the space with underscore.

The following replaces space in all the *.mp3 files with _

$ find . -type f -iname “*.mp3″ -exec rename “s/ /_/g” {} \;

Example 15: Executing two find commands at the same time

As shown in the examples of the find command in its manual page, the following is the syntax which can be used to execute two commands in single traversal.

The following find command example, traverse the filesystem just once, listing setuid files and directories into /root/suid.txt and large files into /root/big.txt.

# find /    \( -perm -4000 -fprintf /root/suid.txt '%#m %u %p\n' \) , \
 \( -size +100M -fprintf /root/big.txt '%-10s %p\n' \)

Our Other 15 Example Series Articles

If you need to return, bookmark this page at del.icio.us for handy reference.

Mommy, I found it! — 15 Practical Linux Find Command Examples

Apart from the basic operation of looking for files under a directory structure, you can also perform several practical operations using find command that will make your command line journey easy.
In this article, let us review 15 practical examples of Linux find command that will be very useful to both newbies and experts.

First, create the following sample empty files under your home directory to try some of the find command examples mentioned below.

# vim create_sample_files.sh
touch MybashProgram.sh
touch mycprogram.c
touch MyCProgram.c
touch Program.c

mkdir backup
cd backup

touch MybashProgram.sh
touch mycprogram.c
touch MyCProgram.c
touch Program.c

# chmod +x create_sample_files.sh

# ./create_sample_files.sh

# ls -R
.:
backup                  MybashProgram.sh  MyCProgram.c
create_sample_files.sh  mycprogram.c      Program.c

./backup:
MybashProgram.sh  mycprogram.c  MyCProgram.c  Program.c

1. Find Files Using Name

This is a basic usage of the find command. This example finds all files with name — MyCProgram.c in the current directory and all its sub-directories.

# find -name "MyCProgram.c"
./backup/MyCProgram.c
./MyCProgram.c

2. Find Files Using Name and Ignoring Case

This is a basic usage of the find command. This example finds all files with name — MyCProgram.c (ignoring the case) in the current directory and all its sub-directories.

# find -iname "MyCProgram.c"
./mycprogram.c
./backup/mycprogram.c
./backup/MyCProgram.c
./MyCProgram.c

3. Limit Search To Specific Directory Level Using mindepth and maxdepth

Find the passwd file under all sub-directories starting from root directory.

# find / -name passwd
./usr/share/doc/nss_ldap-253/pam.d/passwd
./usr/bin/passwd
./etc/pam.d/passwd
./etc/passwd

Find the passwd file under root and one level down. (i.e root — level 1, and one sub-directory — level 2)

# find -maxdepth 2 -name passwd
./etc/passwd

Find the passwd file under root and two levels down. (i.e root — level 1, and two sub-directories — level 2 and 3 )

# find / -maxdepth 3 -name passwd
./usr/bin/passwd
./etc/pam.d/passwd
./etc/passwd

Find the password file between sub-directory level 2 and 4.

# find -mindepth 3 -maxdepth 5 -name passwd
./usr/bin/passwd
./etc/pam.d/passwd

4. Executing Commands on the Files Found by the Find Command.

In the example below, the find command calculates the md5sum of all the files with the name MyCProgram.c (ignoring case). {} is replaced by the current file name.

# find -iname "MyCProgram.c" -exec md5sum {} \;
d41d8cd98f00b204e9800998ecf8427e  ./mycprogram.c
d41d8cd98f00b204e9800998ecf8427e  ./backup/mycprogram.c
d41d8cd98f00b204e9800998ecf8427e  ./backup/MyCProgram.c
d41d8cd98f00b204e9800998ecf8427e  ./MyCProgram.c

5. Inverting the match.

Shows the files or directories whose name are not MyCProgram.c .Since the maxdepth is 1, this will look only under current directory.

# find -maxdepth 1 -not -iname "MyCProgram.c"
.
./MybashProgram.sh
./create_sample_files.sh
./backup
./Program.c

6. Finding Files by its inode Number.

Every file has an unique inode number, using that we can identify that file. Create two files with similar name. i.e one file with a space at the end.

# touch "test-file-name"

# touch "test-file-name "
[Note: There is a space at the end]

# ls -1 test*
test-file-name
test-file-name

From the ls output, you cannot identify which file has the space at the end. Using option -i, you can view the inode number of the file, which will be different for these two files.

# ls -i1 test*
16187429 test-file-name
16187430 test-file-name

You can specify inode number on a find command as shown below. In this example, find command renames a file using the inode number.

# find -inum 16187430 -exec mv {} new-test-file-name \;

# ls -i1 *test*
16187430 new-test-file-name
16187429 test-file-name

You can use this technique when you want to do some operation with the files which are named poorly as shown in the example below. For example, the file with name — file?.txt has a special character in it. If you try to execute “rm file?.txt”, all the following three files will get removed. So, follow the steps below to delete only the “file?.txt” file.

# ls
file1.txt  file2.txt  file?.txt

Find the inode numbers of each file.

# ls -i1
804178 file1.txt
804179 file2.txt
804180 file?.txt

Use the inode number to remove the file that had special character in it as shown below.

# find -inum 804180 -exec rm {} \;

# ls
file1.txt  file2.txt
[Note: The file with name "file?.txt" is now removed]

7. Find file based on the File-Permissions

Following operations are possible.

  • Find files that match exact permission
  • Check whether the given permission matches, irrespective of other permission bits
  • Search by giving octal / symbolic representation

For this example, let us assume that the directory contains the following files. Please note that the file-permissions on these files are different.

# ls -l
total 0
-rwxrwxrwx 1 root root 0 2009-02-19 20:31 all_for_all
-rw-r--r-- 1 root root 0 2009-02-19 20:30 everybody_read
---------- 1 root root 0 2009-02-19 20:31 no_for_all
-rw------- 1 root root 0 2009-02-19 20:29 ordinary_file
-rw-r----- 1 root root 0 2009-02-19 20:27 others_can_also_read
----r----- 1 root root 0 2009-02-19 20:27 others_can_only_read

Find files which has read permission to group. Use the following command to find all files that are readable by the world in your home directory, irrespective of other permissions for that file.

# find . -perm -g=r -type f -exec ls -l {} \;
-rw-r--r-- 1 root root 0 2009-02-19 20:30 ./everybody_read
-rwxrwxrwx 1 root root 0 2009-02-19 20:31 ./all_for_all
----r----- 1 root root 0 2009-02-19 20:27 ./others_can_only_read
-rw-r----- 1 root root 0 2009-02-19 20:27 ./others_can_also_read

Find files which has read permission only to group.

# find . -perm g=r -type f -exec ls -l {} \;
----r----- 1 root root 0 2009-02-19 20:27 ./others_can_only_read

Find files which has read permission only to group [ search by octal ]

# find . -perm 040 -type f -exec ls -l {} \;
----r----- 1 root root 0 2009-02-19 20:27 ./others_can_only_read

8. Find all empty files (zero byte file) in your home directory and its subdirectory

Most files of the following command output will be lock-files and place holders created by other applications.

# find ~ -empty

List all the empty files only in your home directory.

# find . -maxdepth 1 -empty

List only the non-hidden empty files only in the current directory.

# find . -maxdepth 1 -empty -not -name ".*"

9. Finding the Top 5 Big Files

The following command will display the top 5 largest file in the current directory and its subdirectory. This may take a while to execute depending on the total number of files the command has to process.

# find . -type f -exec ls -s {} \; | sort -n -r | head -5

10. Finding the Top 5 Small Files

Technique is same as finding the bigger files, but the only difference the sort is ascending order.

# find . -type f -exec ls -s {} \; | sort -n  | head -5

In the above command, most probably you will get to see only the ZERO byte files ( empty files ). So, you can use the following command to list the smaller files other than the ZERO byte files.

# find . -not -empty -type f -exec ls -s {} \; | sort -n  | head -5

11. Find Files Based on file-type using option -type

Find only the socket files.

# find . -type s

Find all directories

# find . -type d

Find only the normal files

# find . -type f

Find all the hidden files

# find . -type f -name ".*"

Find all the hidden directories

# find -type d -name ".*"

12. Find files by comparing with the modification time of other file.

Show files which are modified after the specified file. The following find command displays all the files that are created/modified after ordinary_file.

# ls -lrt
total 0
-rw-r----- 1 root root 0 2009-02-19 20:27 others_can_also_read
----r----- 1 root root 0 2009-02-19 20:27 others_can_only_read
-rw------- 1 root root 0 2009-02-19 20:29 ordinary_file
-rw-r--r-- 1 root root 0 2009-02-19 20:30 everybody_read
-rwxrwxrwx 1 root root 0 2009-02-19 20:31 all_for_all
---------- 1 root root 0 2009-02-19 20:31 no_for_all

# find -newer ordinary_file
.
./everybody_read
./all_for_all
./no_for_all

13. Find Files by Size

Using the -size option you can find files by size.

Find files bigger than the given size

# find ~ -size +100M

Find files smaller than the given size

# find ~ -size -100M

Find files that matches the exact given size

# find ~ -size 100M

Note: – means less than the give size, + means more than the given size, and no symbol means exact given size.

14. Create Alias for Frequent Find Operations

If you find some thing as pretty useful, then you can make it as an alias. And execute it whenever you want.

Remove the files named a.out frequently.

# alias rmao="find . -iname a.out -exec rm {} \;"
# rmao

Remove the core files generated by c program.

# alias rmc="find . -iname core -exec rm {} \;"
# rmc

 

15. Remove big archive files using find command

The following command removes *.zip files that are over 100M.

# find / -type f -name *.zip -size +100M -exec rm -i {} \;"

Remove all *.tar file that are over 100M using the alias rm100m (Remove 100M). Use the similar concepts and create alias like rm1g, rm2g, rm5g to remove file size greater than 1G, 2G and 5G respectively.

# alias rm100m="find / -type f -name *.tar -size +100M -exec rm -i {} \;"
# alias rm1g="find / -type f -name *.tar -size +1G -exec rm -i {} \;"
# alias rm2g="find / -type f -name *.tar -size +2G -exec rm -i {} \;"
# alias rm5g="find / -type f -name *.tar -size +5G -exec rm -i {} \;"

# rm100m
# rm1g
# rm2g
# rm5g

 

Free up Mac memory 释放内存

Mac 系统用时间长了,Free 的内存越来越少,最后开始读写磁盘上的虚拟内存后,系统会变得很慢,最简单的现象,就是中文输入法,输入一个字母,等半天才会显示候选字。

判断读写虚拟内存有一个简单的方法,打开 Activity Monitor,看 Page outs 的数值。

如果是 0,或者是几十 MB,那说明基本未使用虚拟内存。

如果已经变成几个 G 了,那说明系统已经大量读写虚拟内存,性能已经很慢了。

今天发现一个快速回收内存的方法,只需一条命令,不必再重启电脑:

 leakon@Leakon-MacBook:~$ purge

稍等几秒即可看到结果(其间电脑可能会变成死机一样的状态,不必着急,过一会就好)。

之后,可以看到 蓝色 的 Inactive 内存已经大量回收,绿色的 Free 又回来啦!

free-up-memory-mac

Tags: , ,

5 Solutions to Url Encoded Slashes (%2F) Problem in Apache

Script generated url structure based on actual data coming from external sources such as user input and rss feed, may result in a “404 (Not Found)” error in Apache especially when it contains url encoded slashes. Here’s why…

By default, Apache http web server immediately returns a “404 (Not Found)” error when it encounters url encoded versions of path separators or slashes in URL (Uniform Resource Locator). These are %2F for forward slash (/) and %5C for back slash (\).

The server simply rejects the URL without invoking mod_proxy or mod_rewrite.

This behavior was made to prevent a CGI security hole where sensitive files are mistakenly accessed by scripts using PATH_INFO as a result of poor or no security checks at all.

While this was a legitimate thing to do, this causes problems with some scripts that depend on receiving encoded values which may contain slashes, in URLs.

Here are five (5) ways to solve the problem.

  1. Turn on “AllowEncodedSlashes” directive in Apache

    This directive may be set in server config file (e.g. httpd.conf) and may appear inside <VirtualHost> containers to affect certain websites. Using it in .htaccess files is not allowed.

    <VirtualHost *:80>
        AllowEncodedSlashes On
    </VirtualHost>

    Turning on this directive tells the web server to allow encoded slashes in URLs.

    Example:

    http://www.example.com/books/the_lamp%2C_linux%2Fapache%2Fmysql%2Fphp_solution.html

    One advantage of using this solution is that you don’t have to modify anything in your scripts to process the URL successfully.

    Unfortunately, this option is not always available. Some shared hosting services do not allow access to their server config files, and some might even have Apache installations older than 2.0.46.

    See the Apache HTTP server documentation for allowencodedslashes directive for more info.

    Anyway, there are better alternatives and even more appropriate for dealing with the problem. Though, they require knowledge in PHP scripting and url encodings, implementing them will make your web site less dependent on web server environment.

  2. Replace %2F with %252F and %5C with %255C after url encoding

    Consider the PHP code below.

    $url = ‘http://www.example.com/books/’;

    $title = ‘the lamp, linux/apache/mysql/php solution’;

    // replace all spaces with underscores (_) in the $title

    $title = str_replace(‘ ‘, ‘_’, $title);

    // urlencode the $title and replace all occurrences of encoded slashes in one line

    $url .= str_replace(array(‘%2F’,’%5C’), array(‘%252F’,’%255C’), urlencode($title)) . ‘.html’;

    The url will become:

    http://www.example.com/books/the_lamp%2C_linux%252Fapache%252Fmysql%252Fphp_solution.html

    %25 is actually the url encoded equivalent of percent (%) so the code above is actually url encoding the slashes twice.

  3. Double urlencode()

    Using the same variables in #2, the code below,

    $url .= urlencode(urlencode($title)) . &#39.html’

    will result in a url

    http://www.example.com/books/the_lamp%252C_linux%252Fapache%252Fmysql%252Fphp_solution.html

    Note that other url encoded characters are transformed as well. Notice the %252C. It is changed from %2C which is the encoded equivalent of comma (,).

    Both #2 and #3 will work but they will make your URLs longer, ugly and harder to read.

  4. Use unencoded slashes

    As a good practice, we urlencode strings which may contain slashes, for use in a url. To get back the slashes, we need to replace %2F with slash (/) character.

    Using same variables in #2, the following code,

    $url .= str_replace(&#39%2F&#39, &#39/&#39, urlencode($title)) . &#39.html’

    will output

    http://www.example.com/books/the_lamp%2C_linux/apache/mysql/php_solution.html

    A very nice looking slashes in the URL. However, it reads as though you have a page named “php_solution.html” inside the mysql folder. And, in fact, if you happen to have one, that page will be displayed instead.

  5. Replace slashes with underscores (_)

    The safest and most practical way of handling slashes, is to replace them with underscores before url encoding.

    Using the variables in #2, the following PHP code (also replaces commas),

    $url .= urlencode(str_replace(array(&#39/&#39,&#39,&#39), &#39_&#39, $title)) . &#39.html’

    will output

    http://www.example.com/books/the_lamp__linux_apache_mysql_php_solution.html

    A more meaningful URL.

The solutions mentioned above may not be applicable to the project you are currently working on. But remember, when selecting a solution to implement, choose the one that works best with your development platform and style. And, keep things simple.

Bonus Tips

  1. To capture the title part of the output url above using mod_rewrite:

    RewriteRule ^books/(.*?)\.html$ /index.php?title=$1

  2. A quick method for checking whether a web site is running on Apache http server without looking at the server header field:
    1. Open a browser window, and enter the address http://www.apache.org// (two forward slashes at the end). You will get the home page.
    2. Replace the forward slash at the end with %2F (url encoded slash) so address becomes http://www.apache.org/%2F. The web server will now respond with a “404 (Not Found)” response code!

Proftpd Time zone 时区设置

使用 FTP 连接到服务器修改文件,发现修改时间总是差 8 个小时。比如我 10:28 修改的,显示时间是 2:28。

2:28 是 GMT 时间,北京是 GMT+8:00,东八区,也就是说,应该比 GTM 提前 8 小时才对。

一直不知道怎么设置,后来经过反复尝试,找到了最终方法。

需要 2 步:

(1)设置操作系统为正确时区

/etc/localtime -> /usr/share/zoneinfo/Asia/Shanghai

(2)设置 proftpd 的配置文件,必须把这 2 行放到最后才可以
VIM: /usr/local/etc/proftpd.conf

TimesGMT off # 不使用 GMT 时间
SetEnv TZ :/etc/localtime    # 改用系统时区显示时间

重启即可:

pkill proftpd

/usr/local/sbin/proftpd -c /usr/local/etc/proftpd.conf

Tags: ,

服务器 CPU 性能测试 命令行工具

测试服务器性能

操作系统是 CentOS

 

#!/bin/bash

CONCURRENCY=8
TIME_START=`date`
for ((idx = 0; idx < $CONCURRENCY; idx++))
do
    time echo "scale=5000; a(1)*4" | bc -l > /dev/null &
done

echo    "$TIME_START Start"
       
RUNNING_PROCESS_NUM=1
while [ $RUNNING_PROCESS_NUM -gt 0 ]
do
    printf  ".";
    sleep 1
    RUNNING_PROCESS_NUM=$(ps ax | grep ' bc -l' | grep -v 'grep' | wc -l)
done

TIME_STOP=`date`

echo    "$TIME_STOP Stop"

 

输出是这个样子:
 

Fri Nov 16 14:55:48 CST 2012 Start
...........................................................................................................
real    2m23.185s
user    0m35.672s
sys     0m0.003s
.
real    2m23.394s
user    0m35.712s
sys     0m0.004s

real    2m23.537s
user    0m35.738s
sys     0m0.004s

real    2m23.552s
user    0m35.753s
sys     0m0.004s

real    2m23.557s
user    0m35.775s
sys     0m0.005s

real    2m23.563s
user    0m35.807s
sys     0m0.005s

real    2m23.624s
user    0m35.853s
sys     0m0.000s

real    2m23.748s
user    0m35.888s
sys     0m0.002s
Fri Nov 16 14:58:13 CST 2012 Stop

解释一下我理解的输出。

real 就是从开始,到所有进程结束,实际花费的总时间。
user 是一个 CPU 核心,跑完一个进程花费的时间。

我这是跑了 8 个并发进程,平均每个进程的耗时是 35.8 秒,按照这个计算,总耗时应该是 286 秒。

但看我 Start 和 Stop 的实际输出,只用了 2 分 25 秒,跟 real 是吻合的,换算成纯时间,是 145 秒,基本是 286 的一半。

为什么是一半?

因为我的服务器是双核。

怎么看几核 CPU ?

最简单的方法,就是 top 命令,然后按数字 1,就能展开所有 Cpu,数一数就知道是几核了。

也就是,双核处理器会把原本单核处理器所需要的计算时间减少为一半。

为了证实我的观点,我在单核服务器上测试,也在 8 核处理器上测试,结果如下:

1 核心 CPU

Fri Nov 16 15:33:07 CST 2012 Start
..........................................................................................................................................................................................................................
real    4m41.794s
user    0m35.129s
sys     0m0.022s
.
real    4m42.468s
user    0m35.241s
sys     0m0.026s

real    4m42.529s
user    0m35.241s
sys     0m0.026s

real    4m42.788s
user    0m35.187s
sys     0m0.018s

real    4m43.017s
user    0m35.204s
sys     0m0.021s

real    4m42.777s
user    0m35.189s
sys     0m0.029s

real    4m42.802s
user    0m35.217s
sys     0m0.021s

real    4m42.830s
user    0m35.232s
sys     0m0.028s
Fri Nov 16 15:37:51 CST 2012 Stop

4 核心 CPU

Fri Nov 16 15:35:34 CST 2012 Start
.....................................................................................................................................
real    2m31.157s
user    1m13.471s
sys     0m0.086s

real    2m31.468s
user    1m13.749s
sys     0m0.065s
.
real    2m31.985s
user    1m13.807s
sys     0m0.085s

real    2m32.060s
user    1m13.840s
sys     0m0.113s

real    2m32.095s
user    1m13.730s
sys     0m0.081s

real    2m32.149s
user    1m13.910s
sys     0m0.070s

real    2m32.350s
user    1m14.029s
sys     0m0.070s
.
real    2m32.688s
user    1m13.662s
sys     0m0.114s
Fri Nov 16 15:38:08 CST 2012 Stop

8 核心 CPU

Fri Nov 16 15:31:50 CST 2012 Start
................................
real    0m35.736s
user    0m35.343s
sys     0m0.006s
.
real    0m35.779s
user    0m35.386s
sys     0m0.001s

real    0m35.915s
user    0m35.395s
sys     0m0.004s

real    0m35.982s
user    0m35.442s
sys     0m0.004s

real    0m36.131s
user    0m35.434s
sys     0m0.006s

real    0m36.396s
user    0m35.448s
sys     0m0.101s
..
real    0m38.238s
user    0m35.491s
sys     0m0.005s
..........
real    0m48.669s
user    0m35.412s
sys     0m0.012s
Fri Nov 16 15:32:39 CST 2012 Stop

其实我搞这个测试的最初目的,是在考虑单核心 CPU 的 VPS 服务器是否可以满足需要,从 user 这个数值上,可以判断 CPU 单核心的处理能力。在 CPU 运算密集的程序运行时,多核心的优势还是很明显,但前提是你的程序得能分成多个进程,或者你写多线程的。

测试 VPN 的 CPU 运算能力,这个脚本还是很有用的。

我测试得出的结论,user 在 35 秒左右的,CPU 都很棒。

但上面那个 4 核心的结果,user 时间是 1m13s,真有够慢。明显这个 VPS 的 CPU 比较忙,运算能力差,还好是 4 核心,要是单核心就太慢了。

上面这个测试 CPU 脚本的 shell 是我写的,但是命令是从下面这个网站找到的。

http://duguo.org/kb/server/test/simple_linux_benchmark.html

这里还介绍了测试 DISK 和 NETWORK 的命令以及输出结果,我也 copy 了一份备用:

CPU

time echo "scale=5000; a(1)*4" | bc -l > /dev/null

DISK

time dd bs=1M count=5000 if=/dev/zero of=/tmp/dd.test
time dd bs=1M count=5000 if=/tmp/dd.test of=/dev/null
rm -f /tmp/dd.test

NETWORK-SERVER

nc -v -l 2222 > /dev/null

NETWORK-CLIENT

time dd if=/dev/zero bs=1M count=1000 | nc SERVER_HOST_NAME 2222 -v

Sample Output

leakon@server:~$ time echo "scale=5000; a(1)*4" | bc -l > /dev/null
real    0m42.638s
user    0m42.623s
sys 0m0.012s

leakon@server:~$ time dd bs=1M count=5000 if=/dev/zero of=/tmp/dd.test
5000+0 records in
5000+0 records out
5242880000 bytes (5.2 GB) copied, 92.0074 s, 57.0 MB/s
real    1m32.013s
user    0m0.004s
sys 0m6.996s

leakon@server:~$ time dd bs=1M count=5000 if=/tmp/dd.test of=/dev/null
5000+0 records in
5000+0 records out
5242880000 bytes (5.2 GB) copied, 2.79148 s, 1.9 GB/s
real    0m2.797s
user    0m0.188s
sys 0m2.608s

leakon@server:~$ rm -f /tmp/dd.test

leakon@server:~$ time dd if=/dev/zero bs=1M count=1000 | nc 10.235.47.219 2222 -v
Connection to 10.235.47.219 2222 port [tcp/*] succeeded!
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 113.536 s, 9.2 MB/s
real    1m53.590s
user    0m0.948s
sys 0m7.176s

leakon@server:~$ nc -v -l 2222 > /dev/null
Connection from 10.235.47.219 port 2222 [tcp/*] accepted

VIM 替换 \n 换行符

场景:

用 VIM 编辑文本文件,想在每一行的换行之前,添加一个字符,比如 “;”。

想到替换命令:

:%s/\n/;\n/g

结果总是出错。

后来找到解决方案:

:%s/\n/;\r/g

有人总结的结论:

When searching: \n is newline, \r is CR (carriage return = Ctrl-M = ^M)
When replacing: \r is newline, \n is a null byte (0x00).

意思是:

字符串查找时,”\n” 是换行,”\r” 是回车,也就是经常会看到的 ^M(备注-1)。

字符串替换时,”\r” 是换行,’\n” 是空字符(0x00)。

更多细节可以参考 http://vim.wikia.com/wiki/Search_and_replace

备注-1:

清除所有 ^M 的替换命令

:%s/CTRL+V CTRL+M//g

就是 Control 键+V,然后再 Control 键 + M,就变成了 ^M,然后替换为空就可以了。

 

Tags: