After a bit of naughty learning about bash scripts today, I have summarized some functions that can be encountered when using bash scripts. I would like to skip some of the basic definitions and focus more on practice
Declare variables in bash scripts
Declare local variable
We are all familiar with how to declare a variable in bash as follows
1 2 3 4 5 6 7 8 9 10 11 | var <span class="token operator">=</span> <span class="token string">"test"</span> demo_variable <span class="token punctuation">(</span> <span class="token punctuation">)</span> <span class="token punctuation">{</span> var2 <span class="token operator">=</span> <span class="token string">"test variable"</span> <span class="token keyword">echo</span> <span class="token variable">$var</span> <span class="token keyword">echo</span> <span class="token variable">$var2</span> <span class="token punctuation">}</span> demo_variable <span class="token keyword">echo</span> <span class="token variable">$var</span> <span class="token keyword">echo</span> <span class="token variable">$var2</span> |
Result:
1 2 3 4 5 | test test variable test test variable |
This variable will be used throughout the script. Even inside the function, it is still usable or declared inside the function, and outside it is still usable.
However, in some cases if you want to limit the scope of a variable to functions only, we can use the local variable as follows:
1 2 3 4 5 6 7 8 9 10 11 12 13 | try_local_variable <span class="token punctuation">(</span> <span class="token punctuation">)</span> <span class="token punctuation">{</span> local var <span class="token operator">=</span> <span class="token string">"Test local variable"</span> <span class="token keyword">echo</span> <span class="token variable">$var</span> <span class="token punctuation">}</span> try_local_variable <span class="token keyword">if</span> <span class="token punctuation">[</span> -n <span class="token string">" <span class="token variable">$var</span> "</span> <span class="token punctuation">]</span> <span class="token keyword">then</span> <span class="token keyword">echo</span> <span class="token variable">$var</span> <span class="token keyword">else</span> <span class="token keyword">echo</span> <span class="token string">"Parameter not supplied"</span> <span class="token keyword">fi</span> |
Result:
1 2 3 | Test local variable Parameter not supplied |
Get the value of the variable from the string
1 2 3 4 5 | var <span class="token operator">=</span> <span class="token string">"test"</span> var2 <span class="token operator">=</span> <span class="token string">"var"</span> <span class="token keyword">echo</span> <span class="token string">" <span class="token variable">${!var2}</span> "</span> |
Result
1 2 | test |
Alternatively, assign a value to another variable as follows
1 2 3 4 5 6 | var <span class="token operator">=</span> <span class="token string">"test"</span> var2 <span class="token operator">=</span> <span class="token string">"var"</span> var3 <span class="token operator">=</span> <span class="token string">" <span class="token variable">${!var2}</span> "</span> <span class="token keyword">echo</span> <span class="token variable">$var3</span> |
Result
1 2 | test |
Some functions are often used with String
Replace string
1 2 3 4 5 6 7 8 9 10 | var <span class="token operator">=</span> <span class="token string">"test"</span> <span class="token comment"># replace first 't' only</span> bar <span class="token operator">=</span> <span class="token variable">${var/t/h}</span> <span class="token keyword">echo</span> <span class="token variable">$bar</span> <span class="token comment"># replace all 't'</span> bar <span class="token operator">=</span> <span class="token variable">${var//t/h}</span> <span class="token keyword">echo</span> <span class="token variable">$bar</span> |
Result
1 2 3 | hest hesh |
Cut string
1 2 3 4 | var <span class="token operator">=</span> <span class="token string">"this is a test"</span> <span class="token keyword">echo</span> <span class="token string">" <span class="token variable">$var</span> "</span> <span class="token operator">|</span> <span class="token function">cut</span> -d <span class="token string">' '</span> -f 4 <span class="token keyword">echo</span> <span class="token string">" <span class="token variable">$var</span> "</span> <span class="token operator">|</span> <span class="token function">cut</span> --delimiter <span class="token operator">=</span> <span class="token string">' '</span> --fields <span class="token operator">=</span> 4 |
Result
1 2 | test |
Use array in bash script
Browse all elements of the array
1 2 3 4 5 6 | array1 <span class="token operator">=</span> <span class="token punctuation">(</span> 1 2 3 <span class="token punctuation">)</span> <span class="token keyword">for</span> element <span class="token keyword">in</span> <span class="token string">" <span class="token variable">${array1[@]}</span> "</span> <span class="token keyword">do</span> <span class="token keyword">echo</span> <span class="token variable">$element</span> <span class="token keyword">done</span> |
Result
1 2 3 4 | 1 2 3 |
Merge 2 arrays
1 2 3 4 5 6 7 8 | array1 <span class="token operator">=</span> <span class="token punctuation">(</span> 1 2 3 <span class="token punctuation">)</span> array2 <span class="token operator">=</span> <span class="token punctuation">(</span> 4 5 6 <span class="token punctuation">)</span> new_array <span class="token operator">=</span> <span class="token punctuation">(</span> <span class="token string">" <span class="token variable">${array1[@]}</span> "</span> <span class="token string">" <span class="token variable">${array2[@]}</span> "</span> <span class="token punctuation">)</span> <span class="token keyword">for</span> element <span class="token keyword">in</span> <span class="token string">" <span class="token variable">${new_array[@]}</span> "</span> <span class="token keyword">do</span> <span class="token keyword">echo</span> <span class="token variable">$element</span> <span class="token keyword">done</span> |
Result
1 2 3 4 5 6 7 | 1 2 3 4 5 6 |
Add element to array
Add element to the beginning of the array
1 2 3 4 5 6 7 | array <span class="token operator">=</span> <span class="token punctuation">(</span> 2 3 4 <span class="token punctuation">)</span> array <span class="token operator">=</span> <span class="token punctuation">(</span> 1 <span class="token string">" <span class="token variable">${array[@]}</span> "</span> <span class="token punctuation">)</span> <span class="token keyword">for</span> element <span class="token keyword">in</span> <span class="token string">" <span class="token variable">${array[@]}</span> "</span> <span class="token keyword">do</span> <span class="token keyword">echo</span> <span class="token variable">$element</span> <span class="token keyword">done</span> |
Result
1 2 3 4 5 | 1 2 3 4 |
Add element at the end of the array
1 2 3 4 5 6 7 8 9 10 11 12 13 | new_array <span class="token operator">=</span> <span class="token punctuation">(</span> 0 <span class="token punctuation">)</span> new_array <span class="token operator">=</span> <span class="token punctuation">(</span> <span class="token string">" <span class="token variable">${new_array[@]}</span> "</span> 1 <span class="token punctuation">)</span> <span class="token comment"># Cách 1</span> array <span class="token operator">=</span> <span class="token punctuation">(</span> 2 3 4 <span class="token punctuation">)</span> <span class="token keyword">for</span> element <span class="token keyword">in</span> <span class="token string">" <span class="token variable">${array[@]}</span> "</span> <span class="token keyword">do</span> new_array+ <span class="token operator">=</span> <span class="token punctuation">(</span> <span class="token variable">$element</span> <span class="token punctuation">)</span> <span class="token comment"># Cách 2</span> <span class="token keyword">done</span> <span class="token keyword">for</span> element <span class="token keyword">in</span> <span class="token string">" <span class="token variable">${new_array[@]}</span> "</span> <span class="token keyword">do</span> <span class="token keyword">echo</span> <span class="token variable">$element</span> <span class="token keyword">done</span> |
Result
1 2 3 4 5 6 | 0 1 2 3 4 |
Add element to the specified position of the array
To do this need to follow 3 steps as follows
- Gets all elements before index position “x”
- Add an element to the array.Add an element to the array
- Retrieve all elements from index position “x” and later,
For example, the format below: Add element 4 to the index = 2 position of the array
1 2 3 4 5 6 7 | array <span class="token operator">=</span> <span class="token punctuation">(</span> 0 1 3 5 <span class="token punctuation">)</span> array <span class="token operator">=</span> <span class="token punctuation">(</span> <span class="token string">" <span class="token variable">${array[@]:0:2}</span> "</span> 4 <span class="token string">" <span class="token variable">${array[@]:2}</span> "</span> <span class="token punctuation">)</span> <span class="token keyword">for</span> element <span class="token keyword">in</span> <span class="token string">" <span class="token variable">${array[@]}</span> "</span> <span class="token keyword">do</span> <span class="token keyword">echo</span> <span class="token variable">$element</span> <span class="token keyword">done</span> |
Result
1 2 3 4 5 6 | 0 1 4 3 5 |
Remove element from array
We will need to perform 2 steps
- Gets all elements before index position “x”
- Retrieves all elements from index position “x” + “n” or later
For example, the format below: Remove element from position index = 2 of the array
1 2 3 4 5 6 7 | array <span class="token operator">=</span> <span class="token punctuation">(</span> 0 1 3 5 <span class="token punctuation">)</span> array <span class="token operator">=</span> <span class="token punctuation">(</span> <span class="token string">" <span class="token variable">${array[@]:0:2}</span> "</span> <span class="token string">" <span class="token variable">${array[@]:3}</span> "</span> <span class="token punctuation">)</span> <span class="token keyword">for</span> element <span class="token keyword">in</span> <span class="token string">" <span class="token variable">${array[@]}</span> "</span> <span class="token keyword">do</span> <span class="token keyword">echo</span> <span class="token variable">$element</span> <span class="token keyword">done</span> |
Result
1 2 3 4 | 0 1 5 |
Alternatively, you can use another method when using the unset
function as follows
1 2 3 4 5 6 7 | array <span class="token operator">=</span> <span class="token punctuation">(</span> 0 1 3 5 <span class="token punctuation">)</span> unset -v <span class="token string">'array[1]'</span> <span class="token keyword">for</span> element <span class="token keyword">in</span> <span class="token string">" <span class="token variable">${array[@]}</span> "</span> <span class="token keyword">do</span> <span class="token keyword">echo</span> <span class="token variable">$element</span> <span class="token keyword">done</span> |
Result
1 2 3 4 | 0 3 5 |
Another way when you know the exact value of the element in the array you can use the same alternative as replacing the string in the format.
1 2 | array <span class="token operator">=</span> <span class="token punctuation">(</span> <span class="token string">" <span class="token variable">${array[@]/PATTERN/}</span> "</span> <span class="token punctuation">)</span> |
Here, just using 1 /
symbol is enough to replace all map values
1 2 3 4 5 6 7 | array <span class="token operator">=</span> <span class="token punctuation">(</span> 0 1 3 5 1 2 <span class="token punctuation">)</span> array <span class="token operator">=</span> <span class="token punctuation">(</span> <span class="token string">" <span class="token variable">${array[@]/1/}</span> "</span> <span class="token punctuation">)</span> <span class="token keyword">for</span> element <span class="token keyword">in</span> <span class="token string">" <span class="token variable">${array[@]}</span> "</span> <span class="token keyword">do</span> <span class="token keyword">echo</span> <span class="token variable">$element</span> <span class="token keyword">done</span> |
Result
1 2 3 4 5 6 7 | 1 3 5 2 |
Working with files
Suppose we have a file named text.txt with the following content:
1 2 3 | Line number 1 Line number 2 |
Read all lines in a file
1 2 3 4 5 | <span class="token keyword">while</span> IFS <span class="token operator">=</span> <span class="token string">""</span> <span class="token function">read</span> -r line <span class="token operator">||</span> <span class="token punctuation">[</span> -n <span class="token string">" <span class="token variable">$line</span> "</span> <span class="token punctuation">]</span> <span class="token keyword">do</span> <span class="token keyword">echo</span> <span class="token variable">$line</span> <span class="token keyword">done</span> <span class="token operator"><</span> text.txt |
Result
1 2 3 | Line number 1 Line number 2 |
Edit the file
Add to the beginning of the file
1 2 3 4 | filename <span class="token operator">=</span> <span class="token string">"text.txt (echo "</span> New line added <span class="token string">" | cat - <span class="token variable">$filename</span> ) > "</span> <span class="token variable">$filename</span> <span class="token string">".tmp && mv "</span> <span class="token variable">$filename</span> <span class="token string">".tmp "</span> <span class="token variable">$filename</span> " <span class="token function">cat</span> <span class="token variable">$filename</span> |
Result
1 2 3 4 | New line added Line number 1 Line number 2 |
This way we will do it by creating a file with a .tmp extension and then executing the mv
command to turn the tmp file into the original file name.
Add any line of the file
We can use it using the sed
command with the format <line>i<PATTERN>
.
For example below we will add the first line of the file with the content “Line number 0”. Note here that <line>
must be less than or equal to the number of lines in the file
1 2 3 4 | filename <span class="token operator">=</span> <span class="token string">"text.txt sed -i "</span> 1iLine number 0" <span class="token variable">$filename</span> <span class="token function">cat</span> <span class="token variable">$filename</span> |
Result
1 2 3 4 | Line number 0 Line number 1 Line number 2 |
Add to the end of the file
1 2 3 4 | filename <span class="token operator">=</span> <span class="token string">"text.txt echo "</span> Line number 3" <span class="token operator">>></span> <span class="token variable">$filename</span> <span class="token function">cat</span> <span class="token variable">$filename</span> |
Result
1 2 3 4 | Line number 1 Line number 2 Line number 3 |
Note here if we only use 1 >
sign then it means that will create a new file for example as follows
1 2 3 4 | filename <span class="token operator">=</span> <span class="token string">"text.txt echo "</span> Line number 3" <span class="token operator">></span> <span class="token variable">$filename</span> <span class="token function">cat</span> <span class="token variable">$filename</span> |
Result
1 2 | Line number 3 |
Replace a text encountered in the file
We will continue to use it with the sed
command with the following format
1 2 | <span class="token function">sed</span> -i <span class="token string">"s/PATTERN/REPLACE_PATTERN/"</span> filename |
For example, we will replace “Line” with “This is line”.
1 2 3 4 | filename <span class="token operator">=</span> <span class="token string">"text.txt sed -i "</span> s/Line/This is line/" <span class="token variable">$filename</span> <span class="token function">cat</span> <span class="token variable">$filename</span> |
Result
1 2 3 | This is line number 1 This is line number 2 |
Read yaml file from bash script
After a while of searching, luckily I found a link to write github function quite well enough to read yaml file with the following content:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 | <span class="token comment">#!/usr/bin/env bash</span> <span class="token comment"># shellcheck disable=SC1003</span> <span class="token comment"># Based on https://gist.github.com/pkuczynski/8665367</span> parse_yaml <span class="token punctuation">(</span> <span class="token punctuation">)</span> <span class="token punctuation">{</span> local yaml_file <span class="token operator">=</span> <span class="token variable">$1</span> local prefix <span class="token operator">=</span> <span class="token variable">$2</span> local s local w local fs s <span class="token operator">=</span> <span class="token string">'[[:space:]]*'</span> w <span class="token operator">=</span> <span class="token string">'[a-zA-Z0-9_.-]*'</span> fs <span class="token operator">=</span> <span class="token string">" <span class="token variable"><span class="token variable">$(</span> <span class="token keyword">echo</span> @ <span class="token operator">|</span> <span class="token function">tr</span> @ <span class="token string">'34'</span> <span class="token variable">)</span></span> "</span> <span class="token punctuation">(</span> <span class="token function">sed</span> -e <span class="token string">'/- [^“]'</span> <span class="token string">"[^']"</span> <span class="token string">'.*: /s|([ ]*)- ([[:space:]]*)|1-'$'</span> n <span class="token string">''</span> 12 <span class="token operator">|</span> g <span class="token string">' | sed -ne '</span> /^--/s <span class="token operator">|</span> -- <span class="token operator">||</span> g <span class="token punctuation">;</span> s <span class="token operator">|</span> <span class="token string">"|\"|g; s/[[:space:]]*$//g;' -e "</span> / <span class="token comment">#.*["']/!s| #.*||g; /^#/s|#.*||g;" </span> -e <span class="token string">"s|^( <span class="token variable">$s</span> )( <span class="token variable">$w</span> ) <span class="token variable">$s</span> : <span class="token variable">$s</span> "(.*)" <span class="token variable">$s</span> $|1 <span class="token variable">$fs</span> 2 <span class="token variable">$fs</span> 3|p"</span> -e <span class="token string">"s|^( <span class="token variable">$s</span> )( <span class="token variable">$w</span> ) <span class="token variable">${s}</span> [:-] <span class="token variable">$s</span> (.*) <span class="token variable">$s</span> $|1 <span class="token variable">$fs</span> 2 <span class="token variable">$fs</span> 3|p"</span> <span class="token operator">|</span> <span class="token function">awk</span> -F <span class="token string">" <span class="token variable">$fs</span> "</span> <span class="token string">'{ indent = length( <span class="token variable">$1</span> )/2; if (length( <span class="token variable">$2</span> ) == 0) { conj[indent]="+";} else {conj[indent]="";} vname[indent] = <span class="token variable">$2</span> ; for (i in vname) {if (i > indent) {delete vname[i]}} if (length( <span class="token variable">$3</span> ) > 0) { vn=""; for (i=0; i<indent; i++) {vn=(vn)(vname[i])("_")} printf("%s%s%s%s=("%s")n", "'</span> <span class="token string">" <span class="token variable">$prefix</span> "</span> <span class="token string">'",vn, <span class="token variable">$2</span> , conj[indent-1], <span class="token variable">$3</span> ); } }'</span> <span class="token operator">|</span> <span class="token function">sed</span> -e <span class="token string">'s/_=/+=/g'</span> <span class="token operator">|</span> <span class="token function">awk</span> <span class="token string">'BEGIN { FS="="; OFS="=" } /(-|.).*=/ { gsub("-|\.", "_", <span class="token variable">$1</span> ) } { print }'</span> <span class="token punctuation">)</span> <span class="token operator"><</span> <span class="token string">" <span class="token variable">$yaml_file</span> "</span> <span class="token punctuation">}</span> create_variables <span class="token punctuation">(</span> <span class="token punctuation">)</span> <span class="token punctuation">{</span> local yaml_file <span class="token operator">=</span> <span class="token string">" <span class="token variable">$1</span> "</span> local prefix <span class="token operator">=</span> <span class="token string">" <span class="token variable">$2</span> "</span> <span class="token function">eval</span> <span class="token string">" <span class="token variable"><span class="token variable">$(</span> parse_yaml <span class="token string">" <span class="token variable">$yaml_file</span> "</span> <span class="token string">" <span class="token variable">$prefix</span> "</span> <span class="token variable">)</span></span> "</span> <span class="token punctuation">}</span> |
The function parse_yaml
needs two input values, namely the file name and the prefix value. The create_variables
function is to declare variables after parseing the keys in the file. As in the example below we have the database.yml file and the prefix is config_
1 2 3 4 5 6 7 8 9 10 11 12 | default: adapter: mysql2 reconnect: true encoding: utf8mb4 pool: 5 host: local username: root password: 123456 port: - 3306 - 33306 |
We will execute the following command
1 2 3 4 5 6 7 8 | create_variables database.yml <span class="token string">"config_"</span> <span class="token keyword">echo</span> <span class="token variable">$config_default_adapter</span> <span class="token comment"># Được tạo thành từ giá trị prefix, key :default và key key :adapter</span> <span class="token keyword">for</span> element <span class="token keyword">in</span> <span class="token string">" <span class="token variable">${config_default_port[@]}</span> "</span> <span class="token keyword">do</span> <span class="token keyword">echo</span> <span class="token variable">$element</span> <span class="token keyword">done</span> |
Result
1 2 3 4 | mysql2 3306 33306 |
You can use the sample file to try out the cases.
Above are some of the functions I have just learned. Thank you for watching.