Conclusion
Mining content from the web is becoming an increasingly common activity for individuals and organizations seeking to make informed decisions or simply wanting to collect related information on topics of interest. Web mining is a process of automatically crawling (browsing) websites and scraping (extracting) the relevant content from their pages. Search engines are the most well-known examples of the process. This review indicates that there are many legitimate and beneficial reasons for crawling and scraping websites, though anyone who does should do so politely.