One of the really cool features that Contact Lens provides and is natively integrated into the dailer is the automatic call summarization! Woo. Unfortunately it comes with a few downsides, namely:
- At current you cannot modify the summarization prompt, so you get the same general summary for all calls that is relatively short
- It is (at time of writing) only available in the US regions
This leads down the path of thought: could you instead orchestrate this yourself? Obviously, you lose out on some of the native functionality, however by getting greater control you can then use summarization for different use cases, such as creating notes for a CRM
TL;DR
If you want the short version, you can find the code repo on Github here. It provides interfaces for API access, as well as automatically running across all valid Voice contacts.
The prompt can be tuned and refined as needed, but you should be able to see how it works if you understand Typescript.
A Journey of Transcripts
On the surface, this sounds like an easy request. Contact Lens already generates the files, so you could simply pull the transcript from there, and half the work is done. As it always works out however, this isn’t the most viable. The short version it takes in the realm of 4-5 minutes for the file to be generated, which, while perhaps suitable for an automated solution, doesn’t fit with a solution to integrate notes into a CRM that an Agent can monitor.
Okay, that’s fine, Contact Lens added the ability to stream analysis segments. Well, the downside here is you need to collect and collate the data for contacts and then reconstruct it yourself. Whilst certainly viable, it adds a lot of extra components over the top, and it also means some extra storage and running costs. For a large scale environment this would likely be better, but for smaller scale it may be too much added cost.
That leads us to the API driven ListRealtimeContactAnalysisSegments API. This is essentially the JIT version of the stream above, letting you specify the required Contact ID. AWESOME!. When you start investigating this (and if you don’t find the Admin Page like me) you will likely look at the API spec and think “oh, there is a V2 version, surely thats the one I should use”, especially since the V1 is in its own API namespace with only its function. V2 also supports the pre-redacted transcript, so even easier to work with. To cut a long story short, to work with Voice calls you need to use the V1 API, otherwise an error is raised.
Transcript Get…Now what?
Okay, thats the transcript, and we can easily filter out to the correct transcript data as we need. But, as noted above, we now have this in its unredacted form. This may be fine in some environments, but not others, so lets use Comprehend to Detect and then Redact. Unfortunately, there is only an async version of the API to send a document and retrieve the redacted version back, so instead its more straight forward to detect the entities and manually redact them as we need. This is still fine, and in particular suits Javascript since a replace() call will, by default, work on the first match.
With the ability to redact, then the real meat of the process needs to happen: Summarize the actual call. Fortunately, AWS has provided some guidance on this topic in the past (and indeed was the basis for this code in the first place, then heavily modified). There isn’t really much to say here that doesn’t lead to a deep dive on individual models and their parameters, but the main thing is repeatedly testing and refining the prompt to generate an output that we desire.
Fortunately, the Converse API for Bedrock simplifies a lot of the coding logic around calling a model, as you don’t need to worry about all the specifics to make them function, instead focusing more on the code around it. The great news is by this point the hardest stuff is done, and then it just need wrapping appropriately
Now How to Use This
The great news is this solution then allows you to use it for any calls that have occurred recently, whilst failing for older calls, meaning the window of it being useful should an exploit happen to be reduced.
In the code repo two options are provided, both of which have different use cases
- The first is wrapping it in an API. This can be great for embedding into the Agent workspace so they can optionally request the summary so they can control/modify as required should something be incorrect. This is especially handy for those longer calls where details can easily be forgetten by an Agent
- The second is an automatic “on valid” process using Eventbridge. By checking the configuration of calls to have realtime analysis we can, as soon as the Contact disconnects, start the summarization flow and get it stored. This could be handy for an environment, such as Salesforce, where simply getting a summar is more important in partnership with Agent notes
Summary
Ultimately, this starts to give a lot of freedom. As mentioned, the native solution for AWS Connect is awesome and very useful, but it has its limitations (and its hard to doubt that AWS won’t address these at some point), however by bringing in more control we can then refine each component further.
There is lots of directions this could be taken, for example I have worked on integrating an AI model from another Cloud Provider to use for the summarization logic due to regulations, meanwhile you could also use this to instead do other functionality, such as generating small highlights and creating something like a work cloud. The options are yours.
