Mastering Fixed Delays in Automation Scripts

This blog was generated from a tutorial video you can watch here

What is a Fixed Delay?

A fixed delay is precisely what it sounds like: it introduces a predetermined wait time before your script proceeds with its operation. For instance, consider a scenario where you’re scraping data from a website. If that website loads its content after a specific time—like 10 seconds—you’ll need to account for that delay in your automation script.

Imagine visiting a new URL on the same website we’ve been working with. Upon refreshing the page, the content appears only after 10 seconds. The quotes we want to scrape won’t be displayed until that time has elapsed. So, how do we adjust our scripts accordingly?

Adjusting Selectors

Before we get into the delays, it’s crucial to note that our DOM selectors may change if the website’s structure varies. In our case, we’ve identified the relevant classes for the quotes—let’s call it .quote for text and .author for the author name. With this in mind, let’s run our script.

Initially, our browser window pops up and then immediately closes with no data to show. This happens because we haven’t incorporated a delay into our automation script.

Implementing Fixed Delays

There are primarily two approaches to handle this:

Hardcoding a Fixed Delay: This method is straightforward. If you know the content will always take a certain amount of time to load, say 10 seconds, you can hardcode this into your script. Simply add a delay of 10 seconds, and then follow it up with the command to scrape your quotes.
```
await new Promise(resolve => setTimeout(resolve, 10000)); // waits for 10 seconds
```
While this is effective, it does have its downsides. If the website changes and takes longer to load, your script will fail as it isn’t adaptable to these changes.
Dynamic Waiting: Alternatively, a more flexible method is to wait for a specific element to appear on the page. This way, your script remains accurate regardless of content load times. For example, you can wait for the last quote to appear in the DOM:
```
await page.waitForSelector('.quote:nth-child(10)'); // waits for the 10th quote to load
```

This method allows your script to react dynamically to the website’s conditions—if the load time increases, your script waits until the data is available.

Final Script Example

To sum things up, here’s an example of how you might structure your script with both methods:

// Hardcoded delay
await new Promise(resolve => setTimeout(resolve, 10000)); // waits for 10 seconds

// Dynamic wait for quotes
await page.waitForSelector('.quote:nth-child(10)'); // waits for 10th quote to appear

This way, you can ensure that your scraping process is efficient and reliable, regardless of website performance.

Conclusion

Fixed delays are a powerful tool when writing automation scripts. Whether you’re opting for hard-coded waits or dynamic selectors, understanding when and how to implement these techniques can significantly enhance your scripting efficiency.

If you have any questions or suggestions for future tutorials, feel free to leave your comments below. Happy scr