[ICSI_CPT][Spider]Crawl a website structure via wget

Suppose the target is http://192.168.0.253:5001/, you can mirror the website by executing the following command:

# wget --protocol-directories -r http://192.168.0.253:5001/

The command below enables you to detect the website's structure:

# tree ./http/192.168.0.253:5001/

Here is a command filtering out those webpages containing keywords, such as "password" in the following example:

# grep -nR password ./http/192.168.0.253:5001/