::UPDATE::
Read the 2nd post for more information.
Hey everyone!
For those who know me, I like to experiment with different tech, and with some recent downtime waiting on other scripts, I decided to learn a bit of Python. Naturally, the first project I took on was building a script to test my website for potential improvements in security and performance.
I've been at it for a few weeks now, and the script has already helped me make significant changes, especially to my .htaccess file. These updates have not only enhanced security but also boosted site speed. (Though I might have gone a bit too aggressive on caching—ended up dialing it back due to my site’s dynamic nature!) Once I’ve reviewed my updated .htaccess file with Lonestar, I may even recommend it as a new standard since the current default could use some upgrades.
About the Script
If you're familiar with Python, feel free to give this script a try. I’ve included it as an attachment in this post, along with the list of required modules at the top. You’ll find the command line for each one, so setup should be smooth. There are only three modules required.
Here's an example of the output using Apple’s site on PowerShell terminal:
Security Features
The script includes a range of checks designed to help spot vulnerabilities and enhance site security:
For Non-Python Users
I know most of you might be unfamiliar with Python and may not want to install it just for this test. If that’s the case, feel free to send me a PM with a link to your site, and I’ll run the test for you. I’ll provide a text document of the output and add any improvement recommendations as needed. No charge for this as usual, although, if you want to buy me a beer in return, I certainly won’t refuse!
Notes
This isn’t a full tutorial on using the script. You’ll need to know the basics of Python to run it independently. As for me, I’m still learning too—just felt it was time for a change from batch files!
Let me know if you have any questions, and I hope some of you find it useful!
Read the 2nd post for more information.
Hey everyone!
For those who know me, I like to experiment with different tech, and with some recent downtime waiting on other scripts, I decided to learn a bit of Python. Naturally, the first project I took on was building a script to test my website for potential improvements in security and performance.
I've been at it for a few weeks now, and the script has already helped me make significant changes, especially to my .htaccess file. These updates have not only enhanced security but also boosted site speed. (Though I might have gone a bit too aggressive on caching—ended up dialing it back due to my site’s dynamic nature!) Once I’ve reviewed my updated .htaccess file with Lonestar, I may even recommend it as a new standard since the current default could use some upgrades.
About the Script
If you're familiar with Python, feel free to give this script a try. I’ve included it as an attachment in this post, along with the list of required modules at the top. You’ll find the command line for each one, so setup should be smooth. There are only three modules required.
Here's an example of the output using Apple’s site on PowerShell terminal:
Security Features
The script includes a range of checks designed to help spot vulnerabilities and enhance site security:
- Session Management: Identifies insecure cookies (e.g., missing HttpOnly or Secure flags) to help reduce the risk of XSS and session hijacking.
- CSRF Protection: Scans forms for anti-CSRF tokens to ensure forms aren’t vulnerable to cross-site request forgery attacks.
- DNS Security: Checks for crucial DNS records (SPF, DKIM, and DMARC) to prevent email spoofing and phishing.
- Application Layer Security: Looks for weaknesses in your application headers (e.g., Content Security Policy and X-Frame-Options) to prevent common attacks.
- Vulnerable JavaScript Libraries: Scans your site’s JavaScript dependencies for known vulnerabilities. (Not accurate, but informs you on what to look out for and manually check)
- Host Header Injection: Tests the site’s response to manipulated headers to ensure it’s protected from Host Header Injection attacks.
For Non-Python Users
I know most of you might be unfamiliar with Python and may not want to install it just for this test. If that’s the case, feel free to send me a PM with a link to your site, and I’ll run the test for you. I’ll provide a text document of the output and add any improvement recommendations as needed. No charge for this as usual, although, if you want to buy me a beer in return, I certainly won’t refuse!
Notes
This isn’t a full tutorial on using the script. You’ll need to know the basics of Python to run it independently. As for me, I’m still learning too—just felt it was time for a change from batch files!
Let me know if you have any questions, and I hope some of you find it useful!
Last edited by coRpSE on Thu Oct 31, 2024 11:26 am; edited 2 times in total
Okay, a bit of an update. in the last 2 days, I have put in a ton of work and improvements with the help of ChatGTP. It's actually interisting how many times it can be wrong, yet still, there are people that think its the answer to everything. I will make a seperate post on that in due time on my experinces with it thus far. Overall, I pluged in my code into ChatGTP, and asked it for some suggestions, and while arguing with it, I had a few ideas which ChatGTP actually was somewhat useful and I was able to make the script so much better in many ways. After that, I asked ChatGTP to write up a comprehensive rundown of my script, and below is what it had to say after anyalizing my code.
But first, a couple of notes if you decide to use the script.
ChatGTP Compreehensive Analysis:
This tool is a Comprehensive Site Auditor that helps identify vulnerabilities, improve performance, and ensure best practices are followed on your website. It is particularly beneficial for ongoing site maintenance and security monitoring, offering insights that can directly impact user experience, SEO, and protection from potential cyber threats.
Overview of Functions and Their Benefits
Additional Usability Features
Overall Benefits
Using this tool provides a comprehensive and proactive approach to site maintenance. It identifies weaknesses and suggests best practices to improve site security, accessibility, and performance. By running this script regularly, you can:
Overall, this script acts as an automated web audit, allowing you to maintain a high-quality site without manually checking each component. It saves time, offers insights based on actionable data, and helps keep your site in prime condition, ready to face both user expectations and potential security challenges.
Here are some key changes from the last version to this version. Last version, you had to open the file and edit in the URL of the site, then run the script. That is no longer the case, now to run it, open what ever terminal you are going to use, might it be PowerShell, Command Prompt, or using the terminal in VScode, what ever you use, Windows or Linux, it is as simple as just typing in this 1 command:
Of course, you don't need the
for it to work, but like I said in the notest from above, you must lead your URL with
or
I have updated the first post with the new file
If you find any bugs, please report it to me.
But first, a couple of notes if you decide to use the script.
- There are 2 sections where the script will pause asking you if you like to save a text file with information, suce as the list of images without alt tags, or a list of all render blocking resources. I did it like that for just ease of use.
- When inputing the URL, you must lead with
orPlease login to see this link
Get registered or Log inPlease login to see this link
Get registered or Log in
ChatGTP Compreehensive Analysis:
This tool is a Comprehensive Site Auditor that helps identify vulnerabilities, improve performance, and ensure best practices are followed on your website. It is particularly beneficial for ongoing site maintenance and security monitoring, offering insights that can directly impact user experience, SEO, and protection from potential cyber threats.
Overview of Functions and Their Benefits
- URL Validity Check:
Purpose: Validates that the target URL is correctly formatted and accessible.
Benefit: Ensures the script runs on valid URLs, minimizing errors. Prevents misconfigured or unreachable URLs from triggering further checks, saving time and resources. - DNS Security Check:
Purpose: Verifies DNS configurations for vulnerabilities or potential weaknesses.
Benefit: Identifies misconfigured DNS settings that could be exploited in DNS hijacking or other attacks. Regularly checking DNS settings strengthens the site's overall security posture. - Sessions Management Check:
Purpose: Ensures sessions are securely managed, including secure attributes like HttpOnly and Secure.
Benefit: Improves security for user data, reducing the risk of session hijacking. Essential for sites with login or user interaction, safeguarding against unauthorized access. - Rate Limiting Check:
Purpose: Detects whether rate limiting is in place to prevent abuse (e.g., brute-force login attempts).
Benefit: Protects your site from abuse and brute-force attacks by limiting repeated requests. Rate limiting is a crucial defense against automated attacks, helping maintain server performance and user security. - Page Structure Analysis:
Purpose: Checks the HTML structure for SEO and accessibility improvements, including:- Missing alt tags on images
- Presence of H1 tags
- Count of other headings (H2-H6)
- Oversized images that may slow down page load
Benefit: Improves accessibility and SEO. Ensures all images either have alt tags or are exempted (as per your preference for empty alt tags) and verifies page structure for better search engine optimization and user experience. - Missing alt tags on images
- Meta Tags Analysis:
Purpose: Checks for the presence of essential meta tags (title, description, etc.).
Benefit: Helps optimize for search engines by confirming proper use of meta descriptions, keywords, and other tags, which are vital for site indexing and relevance. - External Scripts Analysis:
Purpose: Reviews the usage of external scripts (JavaScript) on the site, including known vulnerabilities in libraries.
Benefit: Mitigates risks of loading outdated or vulnerable JavaScript libraries, which could expose your site to cross-site scripting (XSS) or other attacks. Helps maintain high-security standards by flagging outdated scripts. - SSL/TLS Version Check:
Purpose: Ensures that SSL/TLS protocols are up-to-date and secure.
Benefit: Reduces the risk of man-in-the-middle attacks by ensuring your site doesn’t support deprecated SSL/TLS versions. Essential for data integrity and user trust. - HTTP Methods Check:
Purpose: Verifies which HTTP methods are enabled and if any insecure ones are accessible.
Benefit: Minimizes risk by restricting access to dangerous HTTP methods (e.g., PUT, DELETE) that could enable attackers to upload or delete files maliciously. - Directory Listing Check:
Purpose: Checks for any unprotected directories that expose file listings.
Benefit: Prevents exposure of sensitive files and site structure, reducing the risk of targeted attacks. Securing directory listings is crucial for maintaining confidentiality and control over website content. - SQL Injection Test:
Purpose: Tests for potential SQL injection vulnerabilities in user input handling.
Benefit: Ensures database security by proactively identifying SQL injection risks, protecting sensitive information, and safeguarding user data. - Test Site Speed:
Purpose: Analyzes the site’s loading speed and identifies potential bottlenecks.
Benefit: Helps improve user experience and SEO by ensuring fast load times. A faster site enhances engagement and ranks better in search engine results, especially for mobile users. - Render-blocking Resource Test:
Purpose: The Render-Blocking Resource Check finds CSS and JavaScript files that delay page rendering.
Benefit: Helps you identify files to load asynchronously or defer for faster page speed. It provides optimization tips and can save a list of blocking resources for easy review in Site_Blocking_Resources.txt. - XSS (Cross-Site Scripting) Test:
Purpose: Checks for vulnerabilities where attackers could inject malicious scripts.
Benefit: Ensures safe interaction for users, especially in forms or areas where content may be dynamically displayed. XSS vulnerabilities can lead to data theft, unauthorized actions, and damaged user trust.
Additional Usability Features
- Suggestion Aggregation: Collects improvement suggestions at the end of the scan.
- Benefit: Provides a consolidated list of actionable recommendations, making it easy to address all identified issues systematically.
- Optional Image Alt Tag Report: Allows the user to generate a report on missing alt tags.
- Benefit: Simplifies accessibility improvements by identifying images needing alternative text, which benefits both SEO and visually impaired users.
- Automatic Cache Clearing: Clears the cache at the start, ensuring the most up-to-date site version is analyzed.
- Benefit: Prevents outdated data from affecting test results, ensuring accuracy and reliability in reporting.
Overall Benefits
Using this tool provides a comprehensive and proactive approach to site maintenance. It identifies weaknesses and suggests best practices to improve site security, accessibility, and performance. By running this script regularly, you can:
- Stay Ahead of Security Threats: By identifying vulnerabilities before they’re exploited.
- Enhance User Experience: Through faster load times, accessible design, and stable functionality.
- Boost SEO: By maintaining a site structure, content optimization, and speed that search engines favor.
- Ensure Legal and Compliance Standards: Like GDPR, ADA, etc., by securing user data and meeting accessibility standards.
Overall, this script acts as an automated web audit, allowing you to maintain a high-quality site without manually checking each component. It saves time, offers insights based on actionable data, and helps keep your site in prime condition, ready to face both user expectations and potential security challenges.
Here are some key changes from the last version to this version. Last version, you had to open the file and edit in the URL of the site, then run the script. That is no longer the case, now to run it, open what ever terminal you are going to use, might it be PowerShell, Command Prompt, or using the terminal in VScode, what ever you use, Windows or Linux, it is as simple as just typing in this 1 command:
Code: [ Select all ]
python siteChecker.py https://www.YourSite.com
Of course, you don't need the
Please login to see this link Get registered or Log in |
Please login to see this link Get registered or Log in |
Please login to see this link Get registered or Log in |
If you find any bugs, please report it to me.