Reference reatures
features.deep
DeepFeatures(model, batch_size=128, num_workers=1, device='cpu')
Extracts features using forward pass of pytorch model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model
|
Module
|
Pytorch model used for the feature extraction. |
required |
batch_size
|
int
|
Batch size used for the feature extraction. |
128
|
num_workers
|
int
|
Number of workers used for data loading. |
1
|
device
|
str
|
Select between cuda and cpu devices. |
'cpu'
|
__call__(dataset)
Extract features from input dataset and return them as a new FeatureDataset.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset
|
ImageDataset
|
Extract features from this dataset. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
feature_dataset |
FeatureDataset
|
A FeatureDataset containing the extracted features |
ClipFeatures(model=None, processor=None, batch_size=128, num_workers=1, device='cpu')
Extract features using CLIP model (https://arxiv.org/pdf/2103.00020.pdf). Uses raw images of input ImageDataset (i.e. dataset.transform = None)
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model
|
CLIPModel
|
Uses VIT-L backbone by default. |
None
|
processor
|
Optional[CLIPProcessor]
|
(CLIPProcessor, optional). Uses VIT-L processor by default. |
None
|
batch_size
|
int
|
Batch size used for the feature extraction. |
128
|
num_workers
|
int
|
Number of workers used for data loading. |
1
|
device
|
str
|
Select between cuda and cpu devices. |
'cpu'
|
__call__(dataset)
Extract clip features from input dataset and return them as a new FeatureDataset.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset
|
ImageDataset
|
Extract features from this dataset. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
feature_dataset |
FeatureDataset
|
A FeatureDataset containing the extracted features |
features.local
GlueFactoryExtractor(config, device=None, num_workers=1)
Base class for Gluefactory extractors.
Common configuration of extractors:
1. max_num_keypoints: Maximum number of keypoints to return.
1. detection_threshold: Threshold for keypoints detection (use 0.0 if force_num_keypoints = True).
1. force_num_keypoints: Force to return exactly max_num_keypoints keypoints.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
config
|
dict
|
Configuration dictionary for the model. |
required |
device
|
str | None
|
Select between cuda and cpu devices. |
None
|
num_workers
|
int
|
Number of workers used for data loading. |
1
|
__call__(dataset)
Extract clip features from input dataset and return them as a new FeatureDataset. Gluefactory extractors requires with 3 channel RBG tensors scaled to [0, 1].
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset
|
ImageDataset
|
Extract features from this dataset. |
required |
Returns: feature_dataset (FeatureDataset): A FeatureDataset containing the extracted features
SuperPointExtractor(detection_threshold=0.0, force_num_keypoints=True, max_num_keypoints=256, device=None, **model_config)
Bases: GlueFactoryExtractor
Superpoint keypoints and descriptors.
- Paper: SuperPoint: Self-Supervised Interest Point Detection and Description
- Link: https://arxiv.org/abs/1712.07629
DiskExtractor(detection_threshold=0.0, force_num_keypoints=True, max_num_keypoints=256, device=None, **model_config)
Bases: GlueFactoryExtractor
DISK keypoints and descriptors.
- Paper: DISK: learning local features with policy gradient
- Link: https://arxiv.org/abs/2006.13566
AlikedExtractor(detection_threshold=0.0, force_num_keypoints=True, max_num_keypoints=256, device=None, **model_config)
Bases: GlueFactoryExtractor
ALIKED keypoints and descriptors.
- Paper: ALIKED: A Lighter Keypoint and Descriptor Extraction Network via Deformable Transformation
- Link: https://arxiv.org/abs/2304.03608
SiftExtractor(backend='opencv', detection_threshold=0.0, force_num_keypoints=True, max_num_keypoints=256, device=None, **model_config)
features.memory
DataToMemory
Loads the dataset into memory for quicker access
Ideal for LOFTR, which operates directly on images, because loading images from storage can become a bottleneck when matching all query-database pairs, requiring n_query x n_database image loads.
__call__(dataset)
Loads data from input dataset into array and returns them as a new FeatureDataset.