Open directory
– AntonioSolano/Getty Images

Research has revealed the transient nature of misconfigurations related to directory listings and the potential impact on network security.

A report from Censys examined how long publicly exposed web directories, often stocked with malicious payloads, remain online. Contrary to popular conception, these file systems disappear in roughly twenty-four hours. This gives attackers long enough to deploy malware or grab data before network security teams can detect anomalies.

The study was limited to plain-HTTP sites, which are easiest to uncover and which transmit every file in an unencrypted manner.

To compare the HTTP body of the open directory, the researchers calculated the hash of the body using TLSH, a rolling hash algorithm, which allowed them to compare HTTP content with greater ease.

The researchers measured how closely each directory’s latest TLSH hash matched it's earlier one, assigning scores from 0 (no resemblance) to 100 (identical), and then correlated those similarity ratings with how long the directories stayed active.

Censys SOTIR-OpenDir-Percentile
The percentiles of network lifespans versus lifespans based on content similarity – Censys

“[We found] that open web directories have shorter network lifespans, a median of one day, meaning that open directories generally come online only to disappear a short while later. However, when we examine open directories through a content lens, we find their median lifespan is closer to three days,” said Ariana Mirian, senior security researcher for Censys.

“This means that even if a web directory blips in and out in terms of network visibility, the content doesn’t necessarily change. This distinction is important, as it allows us to understand how much an open directory has changed, a critical component for actor investigations.”

With this sort of speed, attackers build their infrastructure to vanish or change before it can be discovered by a weekly security scan.

The researchers suggest security teams prioritize recognizing the content itself over address lists, switch from manual reviews to automated telemetry that runs daily, and operate on the expectation that what disappears one day may re-emerge the following on an entirely different host.

The Censys discovery flaw follows similar directory misconfigurations, such as a recent campaign deploying malware under the guise of 404 error pages hosted on websites built using Google Sites.