As more enterprises move their workloads to the cloud, gaining visibility into both application and data traffic is becoming a top priority, but enterprises can’t meet this priority.

This is according to new research from Ixia, a monitoring company under Keysight Business, and Dimensional Research, which found that eight out of 10 companies increased their cloud workloads in 2018, but only 20 percent of respondents felt their company was able to “accurately” monitor these environments.

Of this perplexing paradox, Scott Register, VP of product management for cloud and security at Ixia, said "We live in a world where cloud is the new normal, but we're still learning what the new normal means when it comes to visibility, security monitoring, and application performance monitoring."

Dimensional Research conducted a survey, sponsored by Ixia, of 338 IT professionals from organizations ranging in both size and industry. Forty-one percent of those surveyed were from large organizations, 45 percent from mid-size companies, and 14 percent from small companies. These professionals were responsible for the operation, security, development, deployment, and management of cloud applications or infrastructure.

According to the report, companies are aware that increasing visibility into these environments would enhance their business value — 99 percent of respondents saw a direct link between the two. And this is a problem for companies: 87 percent of respondents felt that this lack of visibility was likely hiding security threats, and nearly half felt it has led to application and network performance issues including outages and the inability to deliver against SLAs.

Respondents also said a comprehensive cloud visibility service could help them identify performance degradation and the source of malicious traffic, detect indicators of compromise, monitor traffic at links to network, and load balance monitoring tools.

The research also highlighted issues with the tools being used. Existing monitoring tools aren’t cutting it as 70 percent of participants said that monitoring the public cloud was more difficult than monitoring data centers and private clouds.

According to Register, cloud environments are hard to monitor because they are highly dynamic and users can deploy new infrastructure on demand at any moment. "Today’s workload can be scaled in or scaled out and look entirely different tomorrow. A monitoring solution needs to react autonomously to these changes," he said. "Monitoring solutions must be able to correlate against higher-level metadata."

To gain insights on cloud environments, Register said the most important thing is to ensure there are no blind spots: "That there is no critical data or communication in their public cloud deployment where a performance or security incident can occur which is not observed by the company."

Survey respondents noted that they didn’t have enough data, specifically packet-level data, to sufficiently monitor the public cloud. Specifically, the professionals indicated that there was much more limited access to data packets in cloud environments than other environments.

Compared to 82 percent of respondents saying that they had sufficient data to monitor on-premises environments, only 15 percent said they had sufficient data to monitor the public cloud. This number was 55 percent for private cloud and 15 percent for hybrid-cloud environments.

This is a problem that respondents say having packet-level monitoring could solve. Eighty-six percent said that having this is important in network and application performance monitoring, and 93 percent said it is important for security monitoring as well.

Packet data can give insight into types of information that log data cannot. "You have to actually analyze the packets themselves to get a full understanding of events," said Register. "Log data is like doing an investigation based on eye-witness accounts. It might do the job if it’s the best you can get. Working from packet data is like doing an investigation from a video recording. Details are apparent that would be missing or misrepresented using any other method."