0. Web reconnaissance
Reconnaissance (or Information Gathering or Enumeration) is the first step of penetration testing (pentest), in which the goal is to find as much information as possible about the pentest Website.
In this Recon step, to be efficient, accurate, and save time and effort, the use of built-in tools or self-writing tools is required.
Today I will introduce to you one tool I use recently: Gobuster
Note: Wordlist you can search and download for free on the internet to use.
1. Introduction and installation
A powerful and new tool, preferred on Kali Linux, written in Golang and updated regularly. Gobuster is a tool used to brute force URLs on websites and DNS subdomains. You can view the full source code here.
To install, in Kali Linux, type: apt-get install gobuster
2. Use
Gobuster provides 3 main mode options: DIR, DNS and vHOST
In this article we will learn about the main, commonly used and useful options.
2.1. Dir mode: Scan the URLs of websites according to wordlist
To display the options in this mode, type: gobuster dir -h
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 | Usage: gobuster dir [flags] Flags: -f, --addslash Append / to each request -c, --cookies string Cookies to use for the requests -e, --expanded Expanded mode, print full URLs -x, --extensions string File extension(s) to search for -r, --followredirect Follow redirects -H, --headers stringArray Specify HTTP headers, -H 'Header1: val1' -H 'Header2: val2' -h, --help help for dir -l, --includelength Include the length of the body in the output -k, --insecuressl Skip SSL certificate verification -n, --nostatus Don't print status codes -P, --password string Password for Basic Auth -p, --proxy string Proxy to use for requests [http(s)://host:port] -s, --statuscodes string Positive status codes (will be overwritten with statuscodesblacklist if set) (default "200,204,301,302,307,401,403") -b, --statuscodesblacklist string Negative status codes (will override statuscodes if set) --timeout duration HTTP Timeout (default 10s) -u, --url string The target URL -a, --useragent string Set the User-Agent string (default "gobuster/3.0.1") -U, --username string Username for Basic Auth --wildcard Force continued operation when wildcard found Global Flags: -z, --noprogress Don't display progress -o, --output string Output file to write results to (defaults to stdout) -q, --quiet Don't print the banner and other noise -t, --threads int Number of concurrent threads (default 10) --delay duration Time each thread waits between requests (e.g. 1500ms) -v, --verbose Verbose output (errors) -w, --wordlist string Path to the wordlist |
We will consider the following command, which will also be a general, basic command used when you use Gobuster:
gobuster dir -u 10.10.10.157 -w /home/vt103/wordlist/common.txt -l -t 30 -e -x php -o 10.10.10.157.result.txt
- 10.10.10.157 is the address of 1 machine on hackthebox.eu , you can visit the website, register and play it
- common.txt is the wordlist file I use, it has about 5000 words
And this is the result:
Let’s take a look at the options used:
dir: mode is selected
-u: the URL
-w: address of wordlist file in our device
-l: Print the length body of the response received
-t: number of threads used, default is 10
-e: used to print the result as a whole URL
-x: add an extension to brute force, for example -x php: add .php manually after URL requests
-o: print the results to a file for easy use later, the default will print to / home / because we call gobuster right at / home /
Some other options:
-U: username for website with Basic Authen
-P: password for websites with Basic Authen
-c: assign a cookie to the request
-s: assign HTTP status codes accepted, default is 200,204,301,302,307,401,403
-r: follow redirects
Based on the results of brute-force URLs above, we have the correct URL and the direction of this machine:
2.2 DNS mode: Scan the DNS subdomains of websites
To display the options in this mode, type: gobuster dns -h
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | Usage: gobuster dns [flags] Flags: -d, --domain string The target domain -h, --help help for dns -r, --resolver string Use custom DNS server (format server.com or server.com:port) -c, --showcname Show CNAME records (cannot be used with '-i' option) -i, --showips Show IP addresses --timeout duration DNS resolver timeout (default 1s) --wildcard Force continued operation when wildcard found Global Flags: -z, --noprogress Don't display progress -o, --output string Output file to write results to (defaults to stdout) -q, --quiet Don't print the banner and other noise -t, --threads int Number of concurrent threads (default 10) --delay duration Time each thread waits between requests (e.g. 1500ms) -v, --verbose Verbose output (errors) -w, --wordlist string Path to the wordlist |
The meaning of the options will be slightly different from the dir mode above. We will consider the following statement:
gobuster dns -d facebook.com -w home / vt103 / wordlist / common.txt -t 30 -i -o dnsrecon.txt
Inside:
-d: only domain name, no http: // or https: //
-w: address of wordlist file in our device
-t: Some threads are used, the default is 10
-i: display the target IP address
-o: print the result as a .txt file
And here is the result, how many of these do you already know? : 
2.3 vHost mode: Scan vHost of website
To display the options in this mode, you type: gobuster vhost -h
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | Usage: gobuster vhost [flags] Flags: -c, --cookies string Cookies to use for the requests -r, --followredirect Follow redirects -H, --headers stringArray Specify HTTP headers, -H 'Header1: val1' -H 'Header2: val2' -h, --help help for vhost -k, --insecuressl Skip SSL certificate verification -P, --password string Password for Basic Auth -p, --proxy string Proxy to use for requests [http(s)://host:port] --timeout duration HTTP Timeout (default 10s) -u, --url string The target URL -a, --useragent string Set the User-Agent string (default "gobuster/3.0.1") -U, --username string Username for Basic Auth Global Flags: -z, --noprogress Don't display progress -o, --output string Output file to write results to (defaults to stdout) -q, --quiet Don't print the banner and other noise -t, --threads int Number of concurrent threads (default 10) --delay duration Time each thread waits between requests (e.g. 1500ms) -v, --verbose Verbose output (errors) -w, --wordlist string Path to the wordlist |
This is a mode I rarely use, and even on gitub introduced by Gobuster, this mode is introduced quite sketchy. Readers can install and try it for themselves.
Github example: gobuster vhost -u https://mysite.com -w common-vhosts.txt
3. Conclusion
Reconnaissance requires the use of a lot of different tools and information channels. Gobuster deserves to be in the “arsenal” of anyone who works on CyberSec because of its speed and efficiency. Happy Hacking!