VADET Help

VADET (Visual Attributes Discovery and Exploration Tool) is a demo of a product search engine suitable for fashion e-shops and similar enterprises. VADET tool provides some common e-shop functionalities such as category browsing, displaying detail of an object and saving objects / queries. Furthermore, VADET implements multi-example query search based on an aggregation of global and local visual image descriptors (image patches).

Above, you can see a labeled screenshot of the VADET tool. Content-based category hierarchy are displayed on the left panel. Visual search parameters, queried objects and top-k results are in the central area. Component for storing pre-selected objects (denoted as visual attributes) is on the right sliding panels. This component may be considered as a background for shoping cart or wishlist in the real-world deployment as well as a basis for a data analytics backend component aiming to the visual attributes discovery and data augumentation. Items are added via "Add to VA" button.

Multi-example queries with local image patches

Users can run multi-example visual queries by clicking on "Query" button on any displayed object (below the product's image).
Furthermore, upon selection of query examples, user may further refine the query by clicking on some place within the image photo. For each click, the nearest image patch is selected and highlighted.
Image patch may be de-selected by repeated click on the image within the patch's boundary.

Ranking of resulting objects is based on the weighted average of global similarity (similarity of whole image descriptors) and local similarity (similarity of patches' descriptors with decriptors of other image's patches). The weight is user defined via the slider on the left side (b)
Local similarity of selected patches is evaluated w.r.t. all available patches of other images and subsequently aggregated by one of the following functions according to the user's selection (c).
  • Max: maximal value of similarity for all patch combinations (an OR-like query)
  • Mean: mean value of similarities of all patch combinations (while searching for patterns that overwhelms the whole image)
  • Mean of max: for each selected patch evaluate maximal similarity with other patches, then average. Recommended for queries insensitive to pattern positions.
  • Mean of distance-weighted max. The same aggregation as above, but before max-pooling, similarities are multiplied by the inversed normalized distance of respective patch centres. Suitable for queries, where pattern positions matters.
In some cases, (almost) background-only image patches may interfere with the query evaluation. Therefore, user has the option to omitt patches with too large portion of background (d) from the results.