The Track

3D shape analysis has a historical symbiosis with the cultural heritage domain. The list of contributions of computer graphics and graphics technology in cultural heritage applications is long, and it includes 3D digitization, reconstruction, virtual restoration, immersive interfaces, and so on. Most of these techniques rely on having a suitable 3D representation of archaeological objects, making it possible to process and analyze these objects with computational methods effectively. Nevertheless, some contributions of shape analysis and geometry processing in CH applications have been limited by the relatively small number of items we can use in experimentation.

This track presents our initiative to promote the research of 3D shape analysis methods in CH domains. Our dataset contains digitized versions of archaeological objects from pre-Columbian cultures in Peru, with varied geometry and artistic styles. The real artifacts are in the Josefina Ramos de Cox (JRC) museum in Lima, Perú. The models were scanned as part of a research project funded by the National Agency for Science and Technology in Peru (CONCYTEC) and in conjunction with the Pontifical Catholic University of Peru, which oversees the administration of the mentioned museum.

In this track, we present two retrieval challenges considering two aspects: the shape and the culture. Regarding the shape, archaeologists in the JRC museum classified the scanned objects by shape using specific taxonomies for Peruvian pre-Colombian artifacts. Regarding culture, the JRC museum keeps records of the pre-Colombian cultures to which the artifact belongs. We collected all this metadata for the scanned models, which serves as input for our retrieval tasks.

The proposed challenges have different degrees of complexity. Retrieval by shape is probably the more affordable challenge given that there exist suitable methods in the 3D shape retrieval literature to deal with geometric characterization. Nevertheless, there are cases where the distinction between objects in different classes is barely perceived. On the other hand, retrieval by culture is a more difficult challenge. Models from the same culture can have varied shapes, and probably the most distinguishable characteristic is the combination of geometry and painting style.

The dataset

The dataset consists of 3D scanned models from cultural heritage objects captured in the Josefina Ramos de Cox museum in Lima, Perú. The technology used to acquire the 3D models was a structured-light desktop scanner, which produces high-resolution 3D textured models. We applied a post-processing step to normalize the position by translating the objects' center to the origin of 3D space. We also change the orientation manually, such that shapes are oriented up in the Y-axis. We keep the original scale of models because the scale can be a distinctive feature that differentiates objects. Finally, we simplify the triangular mesh of each shape to have nearly 40,000 triangle faces.

Note:It is worth to mention that our dataset's objects could contain scanning defects and could be non-manifold.

The Challenges

  • Retrieval by Shape.

    The dataset consists of 938 objects classified into eight categories: jar, pitcher, bowl, figurine, basin, pot, plate, and vase. We split the dataset into a collection set (70% of the dataset) and a query set (30% of the dataset). The collection set contains 661 objects, and the query set has 277 objects. The class with the highest number of models is bowl (with 221 models in total), and the class with the lowest number of models is vase (with 34 objects in total). Next figure shows examples of models in each class.

    We deliver the 3D models in OBJ format for the training set and the test set for the competition. Each model contains geometry and textures. Besides, we deliver a classification file only for the training set. Participants must submit a distance matrix that measures the distance between query objects and collection objects. It means the entry M[i,j] of the distance matrix contains the distance between the i-th query object and the j-th collection object. To facilitate identifying an object, we name the files with the index of the corresponding set.

  • Retrieval by Culture.

    The dataset consists of 637 objects classified into six categories: Chancay, Lurin, Maranga, Nazca, Pando, Supe. We split the dataset into a collection set (70%of the dataset) and a test set (30% of the dataset). The collection set contains 448 objects, and the test set has 189 objects. The class with the highest number of models is Lurin (with 455 shapes in total), and the class with the lowest number of models is Nazca (with seven shapes in total). Next figure shows examples of objects in each class. The procedure for the competition is the same than in the challenge described above.

    It is worth noting that, in both challenges, the classes are not balanced. This phenomenon is even more noticeable in the retrieval-by-culture challenge. Nevertheless, we believe these challenges propose a real application for 3D retrieval algorithms; hence one of our goals is to evaluate the robustness of methods against unbalanced data. Therefore, we offer to make a class-by-class evaluation of the retrieval metrics to identify potential issues concerning data balancing.

Downloads

The data for the challenges can be found in the following links. Each archive contains the collection dataset, the testing dataset, and the classification file for the collection dataset.

Results submission

Participants should submit a distance matrix for each run. Up to 5 matrices (per challenge) may be submitted corresponding to different algorithms or a different parameter setting. The matrix distance must be stored in a file (with a white space as separator) and its size must be of Nq x Nc, where Nq is the number of models in the query set and Nc is the number of models in the collection set. The i-th row of the matrix corresponds with the distances from the i-th query ({i}.obj) to every model in the collection set. We consider the same order imposed by the number in the name file for both collection and query set. So entry M[i,j] corresponds to the distance from {i}.obj in the query set to {j}.obj in the collection set.

In addition, participants must report the following information:

  • System specification: CPU (model, speed in GHz, number of CPU's, RAM per CPU in GB). In case participants use GPU, the required information is model, speed in MHz, memory in GB and number of GPU's.
  • Processing time in seconds: In order to properly evaluate the several stages of a 3D retrieval system, participants should make the difference between offline processing (for example, dictionary computation in BoF approaches or neural network training) and online processing (the time to compute distances between a query object and the entire dataset). Please, provide the average query time for the Nq query objects.

Effectiveness evaluation

We will use four measures to compute the effectiveness of algorithms:

  • Mean Average Precision (MAP): Given a query, its average precision is the average of all precision values computed in each relevant object in the retrieved list. Given several queries, the mean average precision is the mean of average precision of each query.
  • Nearest Neighbor (NN): Given a query, it is the precision at the first object of the retrieved list.
  • First Tier (FT): Given a query, it is the precision when C objects have been retrieved, where C is the number of relevant objects to the query.
  • Second Tier (ST): Given a query, it is the precision when 2*C objects have been retrieved, where C is the number of relevant objects to the query.

To Participants

People interested in participating in this track must register by sending an email to isipiran@dcc.uchile.cl. The registration will help to keep track of the contest.

Timeline

  • January 4th, 2021: Track kick-off
  • January 19th, 2021: Deadline for track registration
  • February 26th, 2021: Deadline for submission of results
  • March 15th, 2021: Submission of track paper

Contact

For additional information, please do not hesitate to contact Ivan Sipiran (isipiran@dcc.uchile.cl).

Organizers

  • Ivan Sipiran, Department of Computer Science, University of Chile.
  • Patrick Lazo, Universidad Nacional de San Agustin, Arequipa, Peru.
  • Cristian Lopez, Universidad La Salle, Arequipa, Peru.