The Challenge

The absence of metadata in the scanned documents posed a significant challenge to CFWB’s employees in terms of structuring and finding them. The manual effort required to determine the content and relevance of each document consumed a considerable amount of time. Furthermore, since the internal folder structure was disorganised, employees could not easily find the documents they needed by simply navigating through the folder structure.

Freddy Vanhove, Accompagnateur du changement aux outils numériques of CFWB

The Solution

To address this challenge, CFWB initiated a research project to find a suitable solution for their document management needs. They collaborated with Xenit to explore the use of Artificial Intelligence (AI) and Machine Learning technology to properly ingest the scanned documents and organise internal folder structure.

Xenit proposed a Deep Learning solution to help complete the metadata of the scanned documents and structure the internal folder structure. A natural language processing (NLP) unsupervised clustering approach was also investigated to propose a new structure for the disorganised internal directories.

For the scanned documents, after being analysed, our internal product Inflow was used to ingest the documents into alfresco in a structured manner. Additionally, a search interface was added that enabled users to efficiently search for the scanned documents based on the extracted metadata. This effort resulted in a structured ingestion of the scanned documents while increasing the clarity by enabling users to quickly find the documents they needed.

The disorganisation within the internal folder structure problem was tackled differently. First off, an effort was made by CFWB to make a novel structuring approach with clear rules about how content should be classified within that novel structure. A machine learning model was then trained to understand that novel structure. From our experiments we saw that the model was able to understand the new structure with high confidence. Using this model we were then able to classify documents from the old structure into the new and improved folder structure. The documents were automatically moved into the new and sturdy structure without the need of any human interaction.

As a different approach, unsupervised clustering based on Latent Dirichlet allocation (LDA) was used to cluster documents from within the old disorganised structure. The clustering algorithm created artificial folders and placed documents with a similar content together in these folders. This proposed a new view on the structuring of their content that could be explored.

Result

The research project conducted by CFWB and Xenit provided a promising solution for their document management challenges. However, due to the constraints in resources and the complexity of implementing the proposed solution, CFWB has yet to implement it.

Conclusion

CFWB’s research project with Xenit has highlighted the potential of Deep Learning technology and AI-based approaches in document management. Although the proposed solution has not been implemented, it has provided valuable insights for CFWB’s digital transformation journey. The use of such technology has the potential to enhance CFWB’s document management process, improve overall business operations and save employees’ time and effort.