Verdant Fox
Blog Posts by Date | tag: 'web'
Python web scraping with Playwright
December 14, 2022 / 22 min read / 5,745 , 1 , 0
Web scraping is the concept of programmatically collecting data from a website. This article will discuss using Playwright for python web scraping. The most popular web-scraping packages for python are requests and Beautiful Soup used together. This combination is potent and straightforward to use for most web pages. However, the use case has limitations because the combination relies on making server requests and reading the static HTML returned. It can be challenging to scrape single-page applications (SPAs) or websites where the objects to scrape are only available after some javascript interactions. Playwright circumvents these limitations by interacting with web pages like humans to find the data that needs scraping.
End-to-end website testing with Playwright
July 20, 2022 / 26 min read / 7,324 , 3 , 0
We all know testing our code is important, right? Automated tests can give peace of mind that our code is working as expected and that it continues to work as expected, even as it is refactored. Python has the pytest framework that offers great tools for testing our backend python code. You can check out my blog post, 9 pytest tips and tricks to take your tests to the next level, to get yourself jump-started testing in python. And javascript has several libraries to test front-end code. But in website testing, how can we write automated tests to ensure that our back-end code (be it python or something else) is working with our front-end code (javascript, HTML, and CSS)?
Introducing Playwright, a fast, easy-to-use, and powerful end-to-end browser automation framework. Similar to the Selenium framework, the Playwright framework has tools that allow us to write tests and scripts that act similar to an actual, human website user. And Playwright has API endpoints in javascript, java, .NET, and python! Sound useful? Read on as we use Playwright with python and pytest to write scripts and end-to-end tests for our Connect 4 game web page.
How I built a Connect 4 AI
May 28, 2022 / 36 min read / 10,313 , 5 , 1
I wrote the board game Connect 4 for my website! 😀 Here's a link to it if you want to check it out. The game can be played against another human or an AI opponent. It can even be played AI-vs-AI, which I find kind of fun to watch -- and which made testing the AI out much easier and more scientific (more on that later). This is the story of how I wrote the AI for the game intuitively, through some trial and error. This post was updated with a significantly enhanced AI version 6 a couple days after the initial release.