How To Monitor & Manage API Logging Challenges

As seen on Spiceworks.

Having your cake and eating it, too – solving the API logging challenges.

With the increased use of APIs to connect an organization’s core business systems and data with customers, partners, and third parties, the risk continues to grow for fraud and misuse, says Edward Roberts, VP of marketing, Neosec.Most organizations use APIs.

Today, it is rare to find an application that does not utilize APIs and rarer still to create a new application not utilizing them. Digital business initiatives practically demand using APIs to ensure seamless services and the most up-to-date information. Unfortunately, most of these APIs are unmonitored.
While previous use of APIs has shouldered risk, it has generally been considered low and not a major concern for security or risk teams. Many APIs connected internal systems and have been fully under the control of the business. Some third-party partner systems have also been connected, but these have often been considered cases of sharing specific, selected data between trusted partners so were assumed safe.

Discover Your APIs First

One reason for not monitoring APIs is that many organizations do not know about most APIs in use. APIs come into play with new applications or agreements to link applications or data sources to customers, partners, suppliers, and other necessary third parties. In either case, it’s common for new APIs to get added to an organization’s attack surface without the involvement or knowledge of IT, security, or risk organizations. For this reason, IT and security professionals tend to underestimate it by 50% or more when asked about the number of APIs a company has in use.

Trying to perform an API inventory can be enormously vexing. Most organizations lack the proper tools, know-how, or procedures to conduct a full discovery at any time. Doing this on an ongoing basis is even more difficult. APIs are continually added, and existing ones may go through significant changes without warning, making the task even more difficult.

Knowing all APIs is the starting place. Like all things in security, one cannot secure what you cannot see. APIs cannot be monitored—and managed—if they are invisible to the organization. An entire API inventory enables the ability to log details, as most security and compliance practices would dictate. The standard practice of logging activity has a long history of proven value. Logging these API interactions represents the means for organizations to conduct behavioral monitoring and threat hunting. 

The Importance of Recording API Activity

Logging API interactions enables both to monitor for meaningful anomalies and conduct forensics. Logs enable an investigation to see what happened and when. Logs provide a sort of paper “data” trail for further examination.

The Open Web Application Security Project (OWASPOpens a new window ) identifies API logging as key for “accountability, visibility, incident alerting, and forensics” and a requirement for compliance. CWE-788 identifies insufficient logging as a common deficiency, along with “improper output neutralization for logs” (CWE-117) and “insertion of sensitive information into the log file (CWE-532).

The OWASP ‘Insufficient Logging’ Problem

Insufficient logging is called out in the OWASP API Top 10 list of common mistakes and bad practices. One reason for insufficient logging is the volume and cost of storing the data through a SIEM tool like Splunk. This is more of a business concern rather than a technological one. The data is not too large to be unwieldy or unmanageable, but it may drive significant cost increases. Perhaps these expenses are the new cost of doing business. APIs are core to digital business initiatives involving closer coupling between businesses and suppliers, partners, and customers. APIs are also the “connective tissue” between the microservices and modules that comprise the construction of modern applications. These machine-to-machine connections expose a company’s most valuable assets, making them vulnerable to abuse, misuse, and fraud. Monitoring APIs is increasingly crucial.

Another reason for insufficient logging is concerns over PII and other regulated data or proprietary information that will be captured and expose further vulnerabilities. Some organizations fear the very act of logging will put them out of compliance. As a result, many companies chose the ostrich approach and do not log API interactions.

So the issue is logging in while safeguarding data and avoiding compliance issues. In other words, how can one have its API monitoring “cake” through logging while ensuring that the logs do not violate compliance rules and standards or put confidential data at risk? One possibility is selective logging to ensure that regulated or proprietary data is not entered into logs. But this is not ideal for behavioral analytics, which requires full data sets to detect anomalies from normal usage. Threat hunting and investigations on partial data would be even more problematic. Log data must provide meaningful information about who accessed data or performed an operation, what records were accessed or manipulated, and how. 

Compliance Problems Specific to APIs

The tokenization of API data may be more than just an implementation technique but, rather, part of a comprehensive framework for treating APIs. The compliance concern is not just a matter of potential exposure or lack of protection for data for behavioral analytics. Regulations themselves may create exposure issues. Within finance, for example, one regulation growing around the world is open banking. Currently, open banking is more prevalent in Europe than in North America. It forces financial institutions to expose their core capabilities through APIs. It requires them to allow the different companies that participate in the ecosystem be able to access their APIs, even if they don’t know them. The challenge becomes one of enabling access while retaining visibility and control.

Healthcare is another area where regulations may also force potential data exposure. In the US, insurance companies are required by regulation to expose PHI information for interoperability purposes so that they can access and process details from providers automatically and without a paper trail. That means that providers have to expose their most sensitive data in a way that they can genuinely control precisely how it’s accessed but in a certain way. What happens if these APIs are exploited and abused?

Is Tokenization a Possible Solution?

Another solution may be to log all details while using tokenization to obfuscate sensitive data so that it would make no sense to anyone without the ability to convert the tokens back into accurate data. Tokenization is the well-established process of substituting a sensitive data element, like a credit card or social security number, for a non-sensitive equivalent with no intrinsic or exploitable value or meaning. This approach enables complete logging without compromise or fear of improper exposure. The data would be under the control of each organization.

While more computationally intensive, modern approaches can tokenize data readily without excessive overhead. Even if there is a bottleneck, the logging is not an in-line process so nothing would cause a delay to any process or data exchange being conducted through APIs. In addition, tokenization done this way would not present any barriers or impediments to scalability. Since it is fully protected, the tokenized logs could be stored within the organization’s infrastructure or by some third party. The key is that only the organization can convert the tokens back into understandable data.

With a larger framework for API data, companies could use tokenization to establish more adequate controls over who accesses protected data and provide a recordkeeping mechanism for audits and ongoing enforcement of safeguarding regulated data. 

The API logging challenge is something that all or nearly all organizations will have to reckon with. Using new technologies and establishing procedures and practices will enable ongoing monitoring and help uphold compliance requirements. The question is not to log or not to log. With careful planning and preparation, organizations can have their API monitoring cake and eat it too.



Test Drive the Neosec API Security Solution

One cloud-native platform, fully deployed in minutes, to protect your APIs.

Start Now