Public Procurement procedure view
Explainability notice
GLOSSARY
| Term | Definition |
|---|---|
| Artificial intelligence(AI) | Artificial intelligence (AI) is technology that enables computers and machines to simulate or to imitate human intelligence and problem-solving capabilities. |
| Classification model | A classification model is a type of machine learning model that categorizes or classifies data into predefined classes or labels. It takes input data and predicts which category or class the data belongs to. |
| CPV (Common Procurement Vocabulary) | CPV (Common Procurement Vocabulary) (1) is a structured system of classification codes used in public procurement across the European Union. It helps categorize and describe the subject matter of supply, service, and works contracts for procurement purposes. The CPV classification system is designed to facilitate the publication and comparison of contract notices, making it easier for suppliers and contracting authorities to find relevant procurement opportunities. |
| eTranslation (2) | eTranslation is a neural machine translation service provided by the European Commission. |
| Machine learning (ML) | Machine learning (ML) is a type of artificial intelligence (AI) that allows software applications to ‘learn’ from past practice and feedback and thereby become more correct at predicting outcomes without being explicitly programmed to do so. |
| Model inference | Model inference is the phase where a trained machine learning model is used to make predictions on new data. |
| Model training | Model training is a critical phase in the development of a machine learning model where the model learns to make predictions based on data. |
| Natural Language Processing (NLP) | Natural Language Processing (NLP) is a field of artificial intelligence (AI) that enables computers to analyse and understand human language, both written and spoken. |
| Public procurement procedure view | A ‘Public procurement procedure view’ aggregates information from all TED notices belonging to the same procedure (related notices) and presents it in a single, simplified and user-friendly perspective. The Public procurement procedure is a new concept of presenting procurement data available in TED notices. |
| scikit-learn | scikit-learn is an open-source machine learning library for the Python programming language. It provides various machine learning algorithms, including algorithms to build classification models. |
| TED | TED-tender electronic daily is the website (https://ted.europa.eu) managed by the Publications Office of the European Union on which the Official Journal S (Supplement to the Official Journal of the European Union, or OJ S) is published. Access to TED is free of charge.
All public tenders above specific contract values must be published in the OJ S. |
(2) https://language-tools.ec.europa.eu/
1. Public procurement procedure view
Public Procurement procedure view, available on OP portal, aggregates information from all TED notices belonging to the same procedure (related notices) and presents it in a single, user-friendly perspective. Public procurement procedure view is a new way of presenting procurement data available in TED notices (3).
The simplified view is designed to be easily understood and navigable by users without specialized procurement knowledge, making it accessible to a broader audience.
Figure 1 – Public Procurement Procedure aggregation.
The Public Procurement Procedure view page contains a visual representation of the procedure status and timeline, along with a geographical representation for the places of performance. The procedure details include key attributes for both the overall procedure and each individual lot. Additionally, the display provides direct links to all related TED notices, ensuring easy access to the primary source of information that was used to aggregate the Public Procurement procedure view.
2. What is CPV enrichment?
CPV enrichment is a functionality that aims to improve the findability of public procurement procedures in order to enhance OP Portal user experience by improving the quality of the search results. To do so, the system combines several Artificial Intelligence (AI) techniques such as natural language processing (NLP) and machine learning (ML) to infer new CPVs codes for all the published procedures, based on their content.
The new CPV codes inferred by the system appear near the original CPV codes on the procedure details page. The page makes a clear distinction between the procedure’s original CPVs and the inferred ones, so that the user is informed the latter are generated with the help of AI. Additionally, the inferred CPV codes are integrated into the search facets, improving the user search experience by allowing users to filter procedure based on enhanced codes.
For example if we consider the procedure with the title “Luxembourg-Luxembourg: Transport services by van and mini-van with dedicated driver” and with the description “Transport services by van and mini-van with dedicated driver.”, the original CPV from the TED notice is “60100000 - Road transport services“, and based on the trained model the enrichment engine inferred an additional, more particular one: “60170000 - Hire of passenger transport vehicles with driver”.
3. How does CPV enrichment work?
The CPV enrichment system relies on a classification model trained with the scikit-learn library. The model takes as input the text content of a procedure, and outputs the confidence score of all supported CPV codes. For a given procedure, the following steps are executed to obtain the predicted CPV codes:
- Extraction: relevant content such as procedures titles and description that helps to infer the CPV codes is extracted from the procedure.
- Translation: the procedure content is translated into English using the eTranslation service if the content of the notice is not available in English.
- Preprocessing: the extracted content may contain irrelevant parts, such as symbols, articles, conjunctions, etc. This information doesn’t help to predict CPV codes, and may impact the inference quality, so it is removed from the procedure content.
- Classification: the extracted content is passed to the classification model to predict a confidence score for each CPV code of predefined classes.
- Filtering: to determine which CPV codes should be kept for the procedure, the obtained confidence scores are compared to a threshold value determined during the training of the model. If the confidence score of a CPV code is greater than the threshold, then the code is considered as relevant for the procedure. For example, if the model predicts a confidence score of 0.7 for the CPV code 85000000 and the threshold is 0.6, then the system considers the inferred CPV code as related to the procedure.
To learn how to infer CPV codes, the model has been trained on the complete set of procedures published on the Portal. Multiple configurations of the model are trained on 80% of the data and evaluated to keep the one with the best prediction quality. The remaining 20% of the data are used to test the quality of the model on data not used during training. All inferred CPV codes with confidence score below the configured threshold are disabled to avoid these codes to be incorrectly predicted or used by the system; CPV inference errors or omissions may still occur.
The classification model is trained and tuned manually and the model with best prediction quality is used to automatically infer additional CPVs on newly processed procurement procedures.
4. Which data are used by CPV enrichment?
The CPV enrichment system uses the procedures publicly available on the OP Portal to learn how to infer CPV codes. The current system has been trained on 2.5 million procedures published from 2011 to date.
The following information are extracted from all the procedures used by the system to train the inference model:
- Title of the procedure.
- Short description of the procedure.
- Title of the lots (if the procedure contains lots).
- Short description of the lots (if the procedure contains lots).
- The main and additional CPV codes already included in the procedure.
- The main and additional CPV codes of the lots (if the procedure contains lots)
That information is published based on the transparency principle applicable to all EU policies and legislation. No personal data is used to train and improve the CPV enrichment system.
5. Limitations at the current point in time
- The system can infer CPV codes up to 5 digits (divisions, groups, classes, and categories). It doesn’t support lower levels.
- The system supports only CPV codes for which a good quality of inference has been validated, which represents 61% of the possible CPV codes up to 5 digits.
- The system only supports inference on procedures published from 2011 to date.
6. Disclaimer - liability aspects
It is to be noted that OP Portal contains content created by AI or other automated technologies. Such content is provided for informational purposes only and should not be relied upon for any specific purpose without verification of its accuracy or completeness.
Public procurement procedure view combines several AI techniques to collect data to improve findability of relevant calls for tenders’ notices published on TED to facilitate and assist in search methods.
CPV enrichment functionality and resulting AI generated output, i.e. inferred CPV codes, is fully automated. In some cases, errors may occur due to the complexity of the system or data. The Public procurement procedure view uses exclusively data available in the OJ S published on TED, Tenders Electronic Daily (https://ted.europa.eu ).
Notwithstanding, the only official information relating to calls for tenders and corresponding notices is as published in the Supplement to the EU Official Journal (OJ S). Although all necessary measures were taken to ensure that the content produced by AI technology is of the highest possible quality, its accuracy cannot however be guaranteed.
Therefore, any liability of the Publications Office and of the EU institutions for any errors or omissions in the outcome resulting from applying AI techniques is hereby disclaimed. No responsibility can be assumed for any consequences of relying upon such AI – generated content. The users are advised to use it with caution and further due diligence is recommended.