Skip to content

SeetaResearch / SeetaDet

Go to a project
Commit 89f1ee28 by Ting PAN
Options

Update README.md

1 parent 19c489b6
Inline Side-by-side
Showing with 114 additions and 548 deletions
CHANGES
...@@ -3,9 +3,7 @@ The list of most significant changes made over time in SeetaDet. ...@@ -3,9 +3,7 @@ The list of most significant changes made over time in SeetaDet.
SeetaDet 0.1.0 (20190311) SeetaDet 0.1.0 (20190311)
Recommended docker for Dragon: Dragon Minimum Required (Version 0.3.0.0)
seetaresearch/dragon:0.3.0.0-rc4-cuda9.1-ubuntu16.04
Changes: Changes:
......
README.md
...@@ -3,12 +3,14 @@ ...@@ -3,12 +3,14 @@
## WHAT's SeetaDet? ## WHAT's SeetaDet?
SeetaDet contains many useful object detectors, including R-CNN series, SSD, SeetaDet contains many useful object detectors, including R-CNN series, SSD,
and the recent RetinaNet. We have achieved the same or higher performance than and the recent RetinaNet.
the baseline reported by the original paper.
We have achieved the same or higher performance than the baseline reported by the original paper.
This repository is based on our [Dragon](https://github.com/seetaresearch/Dragon), This repository is based on our [Dragon](https://github.com/seetaresearch/Dragon),
while the style of codes is PyTorch. The torch-style codes help us to simplify the while the style of codes is PyTorch.
hierarchical pipeline of modern detection.
The torch-style codes help us to simplify the hierarchical pipeline of modern detection.
## Installation ## Installation
...@@ -22,10 +24,41 @@ pip install opencv-python Pillow ...@@ -22,10 +24,41 @@ pip install opencv-python Pillow
#### 2. Compile the C Extensions #### 2. Compile the C Extensions
```bash ```bash
cd SeeTADet/compile cd SeetaDet/compile
bash ./make.sh bash ./make.sh
``` ```
## Quick Start
#### Train a detection model
```bash
cd SeetaDet/tools
python train.py --cfg <MODEL_YAML>
```
We have provided the default YAML examples into ``SeetaDet/configs``.
#### Test a detection model
```bash
cd SeetaDet/tools
python test.py --cfg <MODEL_YAML> --exp_dir <EXP_DIR> --iter <ITERATION>
```
Or
```bash
cd SeetaDet/tools
python test_all.py --cfg <MODEL_YAML> --exp_dir <EXP_DIR>
```
#### Export a detection model to ONNX
```bash
cd SeetaDet/tools
python export.py --cfg <MODEL_YAML> --exp_dir <EXP_DIR> --iter <ITERATION>
```
## Resources ## Resources
#### Pre-trained ImageNet models #### Pre-trained ImageNet models
......
configs/faster_rcnn/coco_faster_rcnn_R-101-FPN_1x.yml
...@@ -32,7 +32,7 @@ FRCNN: ...@@ -32,7 +32,7 @@ FRCNN:
ROI_XFORM_METHOD: RoIAlign ROI_XFORM_METHOD: RoIAlign
ROI_XFORM_RESOLUTION: 7 ROI_XFORM_RESOLUTION: 7
TRAIN: TRAIN:
WEIGHTS: '../data/imagenet_models/R-101.Affine.pth' WEIGHTS: '/data/models/imagenet/R-101.Affine.pth'
DATABASE: 'taas:/data/coco_2014_trainval35k_lmdb' DATABASE: 'taas:/data/coco_2014_trainval35k_lmdb'
IMS_PER_BATCH: 2 IMS_PER_BATCH: 2
USE_DIFF: False # Do not use crowd objects USE_DIFF: False # Do not use crowd objects
......
configs/faster_rcnn/coco_faster_rcnn_R-101-FPN_2x.yml
...@@ -32,7 +32,7 @@ FRCNN: ...@@ -32,7 +32,7 @@ FRCNN:
ROI_XFORM_METHOD: RoIAlign ROI_XFORM_METHOD: RoIAlign
ROI_XFORM_RESOLUTION: 7 ROI_XFORM_RESOLUTION: 7
TRAIN: TRAIN:
WEIGHTS: '../data/imagenet_models/R-101.Affine.pth' WEIGHTS: '/data/models/imagenet/R-101.Affine.pth'
DATABASE: 'taas:/data/coco_2014_trainval35k_lmdb' DATABASE: 'taas:/data/coco_2014_trainval35k_lmdb'
IMS_PER_BATCH: 2 IMS_PER_BATCH: 2
USE_DIFF: False # Do not use crowd objects USE_DIFF: False # Do not use crowd objects
......
configs/faster_rcnn/voc_faster_rcnn_R-50-FPN.yml
...@@ -23,7 +23,7 @@ FRCNN: ...@@ -23,7 +23,7 @@ FRCNN:
ROI_XFORM_METHOD: RoIAlign ROI_XFORM_METHOD: RoIAlign
ROI_XFORM_RESOLUTION: 7 ROI_XFORM_RESOLUTION: 7
TRAIN: TRAIN:
WEIGHTS: '../data/imagenet_models/R-50.Affine.pth' WEIGHTS: '/data/models/imagenet/R-50.Affine.pth'
DATABASE: 'taas:/data/voc_0712_trainval_lmdb' DATABASE: 'taas:/data/voc_0712_trainval_lmdb'
IMS_PER_BATCH: 2 IMS_PER_BATCH: 2
BATCH_SIZE: 128 BATCH_SIZE: 128
......
configs/faster_rcnn/voc_faster_rcnn_VGG-16-C4.yml
...@@ -28,7 +28,7 @@ FRCNN: ...@@ -28,7 +28,7 @@ FRCNN:
ROI_XFORM_RESOLUTION: 7 ROI_XFORM_RESOLUTION: 7
MLP_HEAD_DIM: 4096 MLP_HEAD_DIM: 4096
TRAIN: TRAIN:
WEIGHTS: '../data/imagenet_models/VGG16.RCNN.pth' WEIGHTS: '/data/models/imagenet/VGG16.RCNN.pth'
DATABASE: 'taas:/data/voc_0712_trainval_lmdb' DATABASE: 'taas:/data/voc_0712_trainval_lmdb'
RPN_MIN_SIZE: 16 RPN_MIN_SIZE: 16
IMS_PER_BATCH: 2 IMS_PER_BATCH: 2
......
configs/retinanet/coco_retinanet_400_R-50-FPN_1x.yml
...@@ -32,7 +32,7 @@ FPN: ...@@ -32,7 +32,7 @@ FPN:
RPN_MIN_LEVEL: 3 RPN_MIN_LEVEL: 3
RPN_MAX_LEVEL: 7 RPN_MAX_LEVEL: 7
TRAIN: TRAIN:
WEIGHTS: '../data/imagenet_models/R-50.Affine.pth' WEIGHTS: '/data/models/imagenet/R-50.Affine.pth'
DATABASE: 'taas:/data/coco_2014_trainval35k_lmdb' DATABASE: 'taas:/data/coco_2014_trainval35k_lmdb'
IMS_PER_BATCH: 8 IMS_PER_BATCH: 8
SCALES: [400] SCALES: [400]
......
configs/retinanet/coco_retinanet_400_R-50-FPN_4x.yml
...@@ -35,7 +35,7 @@ DROPBLOCK: ...@@ -35,7 +35,7 @@ DROPBLOCK:
DROP_ON: True DROP_ON: True
DECREMENT: 0.000005 # * 20000 = 0.1 DECREMENT: 0.000005 # * 20000 = 0.1
TRAIN: TRAIN:
WEIGHTS: '../data/imagenet_models/R-50.Affine.pth' WEIGHTS: '/data/models/imagenet/R-50.Affine.pth'
DATABASE: 'taas:/data/coco_2014_trainval35k_lmdb' DATABASE: 'taas:/data/coco_2014_trainval35k_lmdb'
IMS_PER_BATCH: 8 IMS_PER_BATCH: 8
SCALES: [400] SCALES: [400]
......
configs/ssd/voc_ssd_300_AirNet-5b.yml
...@@ -29,7 +29,7 @@ SSD: ...@@ -29,7 +29,7 @@ SSD:
STRIDES: [8, 16, 32] STRIDES: [8, 16, 32]
ASPECT_RATIOS: [[1, 2, 0.5], [1, 2, 0.5], [1, 2, 0.5]] ASPECT_RATIOS: [[1, 2, 0.5], [1, 2, 0.5], [1, 2, 0.5]]
TRAIN: TRAIN:
WEIGHTS: '../data/imagenet_models/AirNet.SSD.pth' WEIGHTS: '/data/models/imagenet/AirNet.SSD.pth'
DATABASE: 'taas:/data/voc_0712_trainval_lmdb' DATABASE: 'taas:/data/voc_0712_trainval_lmdb'
IMS_PER_BATCH: 32 IMS_PER_BATCH: 32
TEST: TEST:
......
configs/ssd/voc_ssd_300_VGG-16.yml
...@@ -32,7 +32,7 @@ SSD: ...@@ -32,7 +32,7 @@ SSD:
ASPECT_RATIOS: [[1, 2, 0.5], [1, 2, 0.5, 3, 0.33], [1, 2, 0.5, 3, 0.33], ASPECT_RATIOS: [[1, 2, 0.5], [1, 2, 0.5, 3, 0.33], [1, 2, 0.5, 3, 0.33],
[1, 2, 0.5, 3, 0.33], [1, 2, 0.5], [1, 2, 0.5]] [1, 2, 0.5, 3, 0.33], [1, 2, 0.5], [1, 2, 0.5]]
TRAIN: TRAIN:
WEIGHTS: '../data/imagenet_models/VGG16.SSD.pth' WEIGHTS: '/data/models/imagenet/VGG16.SSD.pth'
DATABASE: 'taas:/data/voc_0712_trainval_lmdb' DATABASE: 'taas:/data/voc_0712_trainval_lmdb'
IMS_PER_BATCH: 32 IMS_PER_BATCH: 32
TEST: TEST:
......
lib/datasets/coco.py deleted 100644 → 0
# ------------------------------------------------------------
# Copyright (c) 2017-present, SeetaTech, Co.,Ltd.
#
# Licensed under the BSD 2-Clause License.
# You should have received a copy of the BSD 2-Clause License
# along with the software. If not, See,
#
# <https://opensource.org/licenses/BSD-2-Clause>
#
# Codes are based on:
#
# <https://github.com/rbgirshick/py-faster-rcnn/blob/master/lib/datasets/coco.py>
#
# ------------------------------------------------------------
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
import os
import sys
import os.path as osp
import numpy as np
import uuid
import json
import cv2
try:
import cPickle
except:
import pickle as cPickle
from lib.datasets.imdb import imdb
# COCO API
from lib.pycocotools.coco import COCO
from lib.pycocotools.cocoeval import COCOeval
from lib.pycocotools.mask import encode as encode_masks
from lib.core.config import cfg
from lib.utils import boxes as box_utils
class coco(imdb):
def __init__(self, image_set, year):
imdb.__init__(self, 'coco_' + year + '_' + image_set)
self._year = year
self._image_set = image_set
self._data_path = cfg.DATA_DIR
self._classes = ('__background__', # always index 0
'person', 'bicycle', 'car', 'motorcycle', 'airplane',
'bus', 'train', 'truck', 'boat', 'traffic light',
'fire hydrant', 'stop sign', 'parking meter', 'bench',
'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant',
'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag',
'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sports ball',
'kite', 'baseball bat', 'baseball glove', 'skateboard', 'surfboard',
'tennis racket', 'bottle', 'wine glass', 'cup', 'fork', 'knife',
'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli',
'carrot', 'hot dog', 'pizza', 'donut', 'cake', 'chair', 'couch',
'potted plant', 'bed', 'dining table', 'toilet', 'tv', 'laptop',
'mouse', 'remote', 'keyboard', 'cell phone', 'microwave', 'oven',
'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors',
'teddy bear', 'hair drier', 'toothbrush')
self._COCO = COCO(self._get_ann_file())
self._class_to_ind = dict(zip(self.classes, range(self.num_classes)))
cats = self._COCO.loadCats(self._COCO.getCatIds())
self._class_to_coco_cat_id = dict(zip([c['name'] for c in cats],
self._COCO.getCatIds()))
self._salt = str(uuid.uuid4())
self.config = {'cleanup': False, 'use_salt': True}
##############################################
# #
# UTILS #
# #
##############################################
def _get_ann_file(self):
prefix = 'instances' if self._image_set.find('test') == -1 else 'image_info'
image_set = 'minival' if self._image_set == 'trainval35k' else self._image_set
return osp.join(self._data_path, 'annotations',
prefix + '_' + image_set + self._year + '.json')
def _get_comp_id(self):
return '_' + self._salt if self.config['use_salt'] else ''
def _get_prefix(self, type='bbox'):
if type == 'bbox': return 'detections_'
elif type == 'segm': return 'segmentations_'
elif type == 'kpt': return 'keypoints_'
return ''
def _get_coco_results_T(self, results_folder, type='bbox'):
# experiments/model_id/results/detections_minival2014_<comp_id>.json
filename = self._get_prefix(type) + self._name + self._get_comp_id() + '.json'
if not os.path.exists(results_folder): os.makedirs(results_folder)
return os.path.join(results_folder, filename)
##############################################
# #
# COCO-BBOX #
# #
##############################################
def _bbox_results_one_category(self, boxes, cat_id, gt_recs):
results = []
ix = 0
for image_name, rec in gt_recs.items():
dets = boxes[ix]; ix += 1
if isinstance(dets, list) and len(dets) == 0: continue
dets = dets.astype(np.float)
scores = dets[:, -1]
xs = dets[:, 0]
ys = dets[:, 1]
ws = dets[:, 2] - xs + 1
hs = dets[:, 3] - ys + 1
results.extend(
[{'image_id': int(image_name.split('_')[-1].split('.')[0]),
'category_id': cat_id,
'bbox': [xs[k], ys[k], ws[k], hs[k]],
'score': scores[k]} for k in range(dets.shape[0])])
return results
def _do_detection_eval(self, res_file, output_dir):
coco_dt = self._COCO.loadRes(res_file)
coco_eval = COCOeval(self._COCO, coco_dt, 'bbox')
coco_eval.evaluate()
coco_eval.accumulate()
self._print_detection_eval_metrics(coco_eval)
eval_file = osp.join(output_dir, 'detection_results.pkl')
with open(eval_file, 'wb') as fid:
cPickle.dump(coco_eval, fid, cPickle.HIGHEST_PROTOCOL)
print('Wrote COCO eval results to: {}'.format(eval_file))
def evaluate_detections(self, all_boxes, gt_recs, output_dir):
res_file = self._write_coco_results(all_boxes, None, gt_recs, output_dir)
# Only do evaluation on non-test sets
if self._image_set.find('test') == -1:
self._do_detection_eval(res_file, output_dir)
# Optionally cleanup results json file
if self.config['cleanup']: os.remove(res_file)
##############################################
# #
# COCO-SEGM #
# #
##############################################
def _encode_masks(self, masks, boxes, im_h, im_w, binary_thresh=0.4):
num_pred = len(boxes)
assert len(masks) == num_pred
mask_image = np.zeros((im_h, im_w, num_pred), dtype=np.uint8, order='F')
# To work around an issue with cv2.resize (it seems to automatically pad
# with repeated border values), we manually zero-pad the masks by 1 pixel
# prior to resizing back to the original image resolution. This prevents
# "top hat" artifacts. We therefore need to expand the reference boxes by an
# appropriate factor.
M = masks[0].shape[0]
scale = (M + 2.0) / M
ref_boxes = box_utils.expand_boxes(boxes, scale)
ref_boxes = ref_boxes.astype(np.int32)
padded_mask = np.zeros((M + 2, M + 2), dtype=np.float32)
for i in range(num_pred):
ref_box = ref_boxes[i, :4]
mask = masks[i]
padded_mask[1:-1, 1:-1] = mask[:, :]
w = ref_box[2] - ref_box[0] + 1
h = ref_box[3] - ref_box[1] + 1
w = np.maximum(w, 1)
h = np.maximum(h, 1)
mask = cv2.resize(padded_mask, (w, h))
mask = np.array(mask > binary_thresh, dtype=np.uint8)
x1 = max(ref_box[0], 0)
y1 = max(ref_box[1], 0)
x2 = min(ref_box[2] + 1, im_w)
y2 = min(ref_box[3] + 1, im_h)
mask_image[y1 : y2, x1 : x2, i] = mask[(y1 - ref_box[1]) : (y2 - ref_box[1]),
(x1 - ref_box[0]) : (x2 - ref_box[0])]
return encode_masks(mask_image)
def _segm_results_one_category(self, boxes, masks, cat_id, gt_recs):
def filter_boxes(dets):
boxes = dets[:, :4]
ws = boxes[:, 2] - boxes[:, 0]
hs = boxes[:, 3] - boxes[:, 1]
keep = np.where((ws >= 1) & (hs >= 1))[0]
return keep
results = []
ix = 0
for image_name, rec in gt_recs.items():
image_id = int(image_name.split('_')[-1].split('.')[0])
dets = boxes[ix].astype(np.float)
msks = masks[ix]; ix += 1
keep = filter_boxes(dets)
rec = gt_recs[image_name]
im_h, im_w = rec['height'], rec['width']
if len(keep) == 0: continue
scores = dets[:, -1]
mask_encode = self._encode_masks(msks[keep], dets[keep, :4], im_h, im_w, cfg.TEST.BINARY_THRESH)
for k in range(dets[keep].shape[0]):
rle = mask_encode[k]
if sys.version_info >= (3,0): rle['counts'] = rle['counts'].decode()
results.append({
'image_id': image_id,
'category_id': cat_id,
'segmentation': rle,
'score': scores[k]})
return results
def _do_segmentation_eval(self, res_file, output_dir):
coco_dt = self._COCO.loadRes(res_file)
coco_eval = COCOeval(self._COCO, coco_dt, 'segm')
coco_eval.evaluate()
coco_eval.accumulate()
self._print_detection_eval_metrics(coco_eval)
eval_file = osp.join(output_dir, 'segmentation_results.pkl')
with open(eval_file, 'wb') as fid:
cPickle.dump(coco_eval, fid, cPickle.HIGHEST_PROTOCOL)
print('Wrote COCO eval results to: {}'.format(eval_file))
def evaluate_segmentations(self, all_boxes, all_masks, gt_recs, output_dir):
res_file = self._write_coco_results(all_boxes, all_masks, gt_recs, output_dir)
# Only do evaluation on non-test sets
if self._image_set.find('test') == -1:
self._do_segmentation_eval(res_file, output_dir)
# Optionally cleanup results json file
if self.config['cleanup']: os.remove(res_file)
##############################################
# #
# EVAL-API #
# #
##############################################
def _write_coco_results(self, all_boxes, all_masks, gt_recs, output_dir):
# <bbox>
# [{"image_id": 42,
# "category_id": 18,
# "bbox": [258.15,41.29,348.26,243.78],
# "score": 0.236}, ...]
# <segmentation>
# [{"image_id": 42,
# "category_id": 18,
# "segmentation": [.....],
# "score": 0.236}, ...]
results = []
eval_type = 'bbox'
if all_masks: eval_type = 'segm'
res_file = self._get_coco_results_T(output_dir, eval_type)
for cls_ind, cls in enumerate(self.classes):
if cls == '__background__':
continue
print('Collecting {} results ({:d}/{:d})'.format(cls, cls_ind, self.num_classes - 1))
coco_cat_id = self._class_to_coco_cat_id[cls]
if all_masks is None:
results.extend(self._bbox_results_one_category(
all_boxes[cls_ind], coco_cat_id, gt_recs))
else:
results.extend(self._segm_results_one_category(
all_boxes[cls_ind], all_masks[cls_ind], coco_cat_id, gt_recs))
print('Writing results json to {}'.format(res_file))
with open(res_file, 'w') as fid: json.dump(results, fid)
return res_file
def _print_detection_eval_metrics(self, coco_eval):
IoU_lo_thresh = 0.5
IoU_hi_thresh = 0.95
def _get_thr_ind(coco_eval, thr):
ind = np.where((coco_eval.params.iouThrs > thr - 1e-5) &
(coco_eval.params.iouThrs < thr + 1e-5))[0][0]
iou_thr = coco_eval.params.iouThrs[ind]
assert np.isclose(iou_thr, thr)
return ind
ind_lo = _get_thr_ind(coco_eval, IoU_lo_thresh)
ind_hi = _get_thr_ind(coco_eval, IoU_hi_thresh)
# precision has dims (iou, recall, cls, area range, max dets)
# area range index 0: all area ranges
# max dets index 2: 100 per image
precision = \
coco_eval.eval['precision'][ind_lo:(ind_hi + 1), :, :, 0, 2]
ap_default = np.mean(precision[precision > -1])
print ('~~~~ Mean and per-category AP @ IoU=[{:.2f},{:.2f}] '
'~~~~'.format(IoU_lo_thresh, IoU_hi_thresh))
print('{:.1f}'.format(100 * ap_default))
for cls_ind, cls in enumerate(self.classes):
if cls == '__background__':
continue
# minus 1 because of __background__
precision = coco_eval.eval['precision'][ind_lo:(ind_hi + 1), :, cls_ind - 1, 0, 2]
ap = np.mean(precision[precision > -1])
print('{:.1f}'.format(100 * ap))
print('~~~~ Summary metrics ~~~~')
coco_eval.summarize()
return coco_eval.prs()
def competition_mode(self, on):
if on:
self.config['use_salt'] = False
self.config['cleanup'] = False
else:
self.config['use_salt'] = True
self.config['cleanup'] = True
\ No newline at end of file
lib/datasets/factory.py
...@@ -13,48 +13,27 @@ ...@@ -13,48 +13,27 @@
# #
# ------------------------------------------------------------ # ------------------------------------------------------------
from lib.datasets.pascal_voc import pascal_voc from lib.datasets.taas import TaaS
from lib.datasets.coco import coco
from lib.datasets.taas import taas
__sets = {}
# pascal voc # TaaS DataSet
for year in ['2007', '2012', '0712']: _GLOBAL_DATA_SETS = {'taas': lambda source: TaaS(source)}
for split in ['train', 'val', 'trainval', 'test']:
name = 'voc_{}_{}'.format(year, split)
__sets[name] = (lambda split=split, year=year: pascal_voc(split, year))
# coco 2014
for year in ['2014']:
for split in ['train', 'val', 'trainval35k', 'minival', 'valminusminival']:
name = 'coco_{}_{}'.format(year, split)
__sets[name] = (lambda split=split, year=year: coco(split, year))
# coco 2015 & 2017
for year in ['2015', '2017']:
for split in ['test', 'test-dev']:
name = 'coco_{}_{}'.format(year, split)
__sets[name] = (lambda split=split, year=year: coco(split, year))
# taas
__sets['taas'] = (lambda source: taas(source))
def get_imdb(name): def get_imdb(name):
"""Get an imdb (image database) by name.""" """Get an imdb (image database) by name."""
keys = name.split(':') keys = name.split(':')
if len(keys) == 2: if len(keys) >= 2:
cls, source = keys cls, source = keys[0], ':'.join(keys[1:])
if cls not in __sets: if cls not in _GLOBAL_DATA_SETS:
raise KeyError('Unknown dataset: {}'.format(cls)) raise KeyError('Unknown dataset: {}'.format(cls))
return __sets[cls](source) return _GLOBAL_DATA_SETS[cls](source)
elif len(keys) == 1: elif len(keys) == 1:
return __sets[name]() return _GLOBAL_DATA_SETS[name]()
else: else:
raise ValueError('Illegal format of image database: {}'.format(name)) raise ValueError('Illegal format of image database: {}'.format(name))
def list_imdbs(): def list_imdbs():
"""List all registered imdbs.""" """List all registered imdbs."""
return __sets.keys() return _GLOBAL_DATA_SETS.keys()
\ No newline at end of file
lib/datasets/pascal_voc.py deleted 100644 → 0
# ------------------------------------------------------------
# Copyright (c) 2017-present, SeetaTech, Co.,Ltd.
#
# Licensed under the BSD 2-Clause License.
# You should have received a copy of the BSD 2-Clause License
# along with the software. If not, See,
#
# <https://opensource.org/licenses/BSD-2-Clause>
#
# Codes are based on:
#
# <https://github.com/rbgirshick/py-faster-rcnn/blob/master/lib/datasets/pascal_voc.py>
#
# ------------------------------------------------------------
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import os
import json
import numpy as np
import uuid
try:
import cPickle
except:
import pickle as cPickle
from .imdb import imdb
from .voc_eval import voc_bbox_eval, voc_segm_eval
class pascal_voc(imdb):
def __init__(self, image_set, year, name='voc'):
imdb.__init__(self, name + '_' + year + '_' + image_set)
self._year = year
self._image_set = image_set
self._classes = ('__background__', # always index 0
'aeroplane', 'bicycle', 'bird', 'boat',
'bottle', 'bus', 'car', 'cat', 'chair',
'cow', 'diningtable', 'dog', 'horse',
'motorbike', 'person', 'pottedplant',
'sheep', 'sofa', 'train', 'tvmonitor')
self._class_to_ind = dict(zip(self.classes, range(self.num_classes)))
self._salt = str(uuid.uuid4())
self.config = {'cleanup': True, 'use_salt': True}
def _get_comp_id(self):
return '_' + self._salt if self.config['use_salt'] else ''
def _get_prefix(self, type='bbox'):
if type == 'bbox': return 'detections_'
elif type == 'segm': return 'segmentations_'
elif type == 'kpt': return 'keypoints_'
return ''
def _get_voc_results_T(self, results_folder, type='bbox'):
# experiments/model_id/results/detections_voc_2007_test_<comp_id>_aeroplane.txt
filename = self._get_prefix(type) + self._name + self._get_comp_id() + '_{:s}.txt'
if not os.path.exists(results_folder): os.makedirs(results_folder)
return os.path.join(results_folder, filename)
def _write_voc_bbox_results(self, all_boxes, gt_recs, output_dir):
for cls_ind, cls in enumerate(self.classes):
if cls == '__background__': continue
print('Writing {} VOC format bbox results'.format(cls))
filename = self._get_voc_results_T(output_dir).format(cls)
with open(filename, 'wt') as f:
ix = 0
for image_id, rec in gt_recs.items():
dets = all_boxes[cls_ind][ix]; ix += 1
if dets == []: continue
for k in range(dets.shape[0]):
f.write('{:s} {:.3f} {:.1f} {:.1f} {:.1f} {:.1f}\n'.
format(image_id, dets[k, -1],
dets[k, 0] + 1, dets[k, 1] + 1,
dets[k, 2] + 1, dets[k, 3] + 1))
def _write_seg_results_file(self, all_boxes, all_masks):
for cls_inds, cls in enumerate(self.classes):
if cls == '__background__': continue
print('Writing {} VOC results file'.format(cls))
results_folder = os.path.join(self._devkit_path, 'results', 'seg')
if not os.path.exists(results_folder): os.makedirs(results_folder)
det_filename = os.path.join(results_folder, cls + '_det.pkl')
seg_filename = os.path.join(results_folder, cls + '_seg.pkl')
with open(det_filename, 'wb') as f:
cPickle.dump(all_boxes[cls_inds], f, cPickle.HIGHEST_PROTOCOL)
with open(seg_filename, 'wb') as f:
cPickle.dump(all_masks[cls_inds], f, cPickle.HIGHEST_PROTOCOL)
def _do_voc_bbox_eval(self, gt_recs, output_dir):
aps = []
# The PASCAL VOC metric changed in 2010
use_07_metric = True if int(self._year) < 2010 else False
print('VOC07 metric? ' + ('Yes' if use_07_metric else 'No') + '\n')
for i, cls in enumerate(self._classes):
if cls == '__background__':
continue
det_file = self._get_voc_results_T(output_dir).format(cls)
rec, prec, ap = voc_bbox_eval(det_file, gt_recs, cls,
IoU=0.5, use_07_metric=use_07_metric)
aps += [ap]
print('AP for {} = {:.4f}'.format(cls, ap))
print('Mean AP = {:.4f}\n'.format(np.mean(aps)))
def _do_voc_segm_eval(self, imagenames, output_dir):
aps = []
# define this as true according to SDS's evaluation protocol
use_07_metric = True
print('VOC07 metric? ' + ('Yes' if use_07_metric else 'No'))
print('~~~~~~ Evaluation use min overlap = 0.5 ~~~~~~')
for i, cls in enumerate(self.classes):
if cls == '__background__':
continue
det_file = os.path.join(output_dir, 'bbox_' + cls + '.pkl')
seg_file = os.path.join(output_dir, 'segm_' + cls + '.pkl')
mask_file = os.path.join(self.cache_path, self.name + '.pkl')
ap = seg_eval_v2(det_file, seg_file, mask_file, imagenames, cls,
ovthresh=0.5, use_07_metric=use_07_metric)
aps += [ap]
print('AP for {} = {:.2f}'.format(cls, ap))
print('Mean AP@0.5 = {:.2f}'.format(np.mean(aps)))
print('~~~~~~ Evaluation use min overlap = 0.7 ~~~~~~')
aps = []
for i, cls in enumerate(self.classes):
if cls == '__background__':
continue
det_file = os.path.join(output_dir, 'bbox_' + cls + '.pkl')
seg_file = os.path.join(output_dir, 'segm_' + cls + '.pkl')
mask_file = os.path.join(self.cache_path, self.name + '.pkl')
ap = seg_eval_v2(det_file, seg_file, mask_file, imagenames, cls,
ovthresh=0.7, use_07_metric=use_07_metric)
aps += [ap]
print('AP for {} = {:.2f}'.format(cls, ap))
print('Mean AP@0.7 = {:.2f}'.format(np.mean(aps)))
def evaluate_detections(self, all_boxes, gt_recs, output_dir):
self._write_voc_bbox_results(all_boxes, gt_recs, output_dir)
self._do_voc_bbox_eval(gt_recs, output_dir)
if self.config['cleanup']:
for cls in self._classes:
if cls == '__background__': continue
filename = self._get_voc_results_T(output_dir).format(cls)
os.remove(filename)
def competition_mode(self, on):
if on:
self.config['use_salt'] = False
self.config['cleanup'] = False
else:
self.config['use_salt'] = True
self.config['cleanup'] = True
\ No newline at end of file
lib/datasets/taas.py
...@@ -35,7 +35,7 @@ from lib.utils import boxes as box_utils ...@@ -35,7 +35,7 @@ from lib.utils import boxes as box_utils
from lib.pycocotools.mask import encode as encode_masks from lib.pycocotools.mask import encode as encode_masks
class taas(imdb): class TaaS(imdb):
def __init__(self, source): def __init__(self, source):
imdb.__init__(self, 'taas') imdb.__init__(self, 'taas')
self._classes = cfg.MODEL.CLASSES self._classes = cfg.MODEL.CLASSES
...@@ -151,6 +151,40 @@ class taas(imdb): ...@@ -151,6 +151,40 @@ class taas(imdb):
############################################## ##############################################
# # # #
# ROT #
# #
##############################################
def _write_voc_rbox_results(self, all_boxes, gt_recs, output_dir):
for cls_ind, cls in enumerate(self.classes):
if cls == '__background__': continue
print('Writing {} VOC format rbox results'.format(cls))
filename = self._get_voc_results_T(output_dir).format(cls)
with open(filename, 'wt') as f:
ix = 0
for image_id, rec in gt_recs.items():
dets = all_boxes[cls_ind][ix]; ix += 1
if dets == []: continue
for k in range(dets.shape[0]):
f.write('{:s} {:.3f} {:.1f} {:.1f} {:.1f} {:.1f} {:.2f}\n'.
format(image_id, dets[k, -1],
dets[k, 0] + 1, dets[k, 1] + 1,
dets[k, 2] + 1, dets[k, 3] + 1, dets[k, 4]))
def _do_voc_rbox_eval(self, gt_recs, output_dir, IoU=0.5, use_07_metric=True):
aps = []
print('VOC07 metric? ' + ('Yes' if use_07_metric else 'No'))
for i, cls in enumerate(self._classes):
if cls == '__background__': continue
det_file = self._get_voc_results_T(output_dir).format(cls)
rec, prec, ap = voc_rbox_eval(det_file, gt_recs, cls,
IoU=IoU, use_07_metric=use_07_metric)
if ap > 0: aps += [ap]
print('AP for {} = {:.4f}'.format(cls, ap))
print('Mean AP = {:.4f}\n'.format(np.mean(aps)))
##############################################
# #
# COCO # # COCO #
# # # #
############################################## ##############################################
...@@ -398,6 +432,15 @@ class taas(imdb): ...@@ -398,6 +432,15 @@ class taas(imdb):
print('~~~~~~ Evaluation IoU@0.7 ~~~~~~') print('~~~~~~ Evaluation IoU@0.7 ~~~~~~')
self._do_voc_bbox_eval(gt_recs, output_dir, self._do_voc_bbox_eval(gt_recs, output_dir,
IoU=0.7, use_07_metric='2007' in protocol) IoU=0.7, use_07_metric='2007' in protocol)
elif 'rot' in protocol:
self._write_voc_rbox_results(all_boxes, gt_recs, output_dir)
if not 'wo' in protocol:
print('\n~~~~~~ Evaluation IoU@0.5 ~~~~~~')
self._do_voc_rbox_eval(gt_recs, output_dir,
IoU=0.5, use_07_metric='2007' in protocol)
print('~~~~~~ Evaluation IoU@0.7 ~~~~~~')
self._do_voc_rbox_eval(gt_recs, output_dir,
IoU=0.7, use_07_metric='2007' in protocol)
elif 'coco' in protocol: elif 'coco' in protocol:
from lib.pycocotools.coco import COCO from lib.pycocotools.coco import COCO
if os.path.exists(cfg.TEST.JSON_FILE): if os.path.exists(cfg.TEST.JSON_FILE):
......
lib/nms/nms_wrapper.py
...@@ -30,11 +30,6 @@ try: ...@@ -30,11 +30,6 @@ try:
except ImportError as e: except ImportError as e:
print('Failed to import gpu nms. Error: {0}'.format(str(e))) print('Failed to import gpu nms. Error: {0}'.format(str(e)))
try:
from lib.utils.rboxes import RNMSWrapper
except ImportError as e:
print('Failed to import rnms. Error: {0}'.format(str(e)))
def nms(detections, thresh, force_cpu=False): def nms(detections, thresh, force_cpu=False):
"""Perform either CPU or GPU Hard-NMS.""" """Perform either CPU or GPU Hard-NMS."""
...@@ -63,19 +58,3 @@ def soft_nms( ...@@ -63,19 +58,3 @@ def soft_nms(
sigma, sigma,
score_thresh, score_thresh,
) )
\
def rnms(detections, thresh):
"""Perform CPU Hard-NMS on rotated boxes.
Parameters
----------
detections : numpy.ndarray
(N, 6) of double [cx, cy, w, h, a, scores]
thresh : float
The nms thresh.
"""
if detections.shape[0] == 0: return []
wrapper = RNMSWrapper()
return wrapper.nms(detections, thresh)
\ No newline at end of file
lib/ssd/data/preprocessing/resize.py
...@@ -13,8 +13,7 @@ from __future__ import absolute_import ...@@ -13,8 +13,7 @@ from __future__ import absolute_import
from __future__ import division from __future__ import division
from __future__ import print_function from __future__ import print_function
import PIL.Image import cv2
import numpy as np
import numpy.random as npr import numpy.random as npr
from lib.core.config import cfg from lib.core.config import cfg
...@@ -25,17 +24,17 @@ class Resizer(object): ...@@ -25,17 +24,17 @@ class Resizer(object):
self._re_height = cfg.SSD.RESIZE.HEIGHT self._re_height = cfg.SSD.RESIZE.HEIGHT
self._re_width = cfg.SSD.RESIZE.WIDTH self._re_width = cfg.SSD.RESIZE.WIDTH
interp_list = { interp_list = {
'LINEAR': PIL.Image.BILINEAR, 'LINEAR': cv2.INTER_LINEAR,
'AREA': PIL.Image.BILINEAR, 'AREA': cv2.INTER_AREA,
'NEAREST': PIL.Image.NEAREST, 'NEAREST': cv2.INTER_NEAREST,
'CUBIC': PIL.Image.CUBIC, 'CUBIC': cv2.INTER_CUBIC,
'LANCZOS4': PIL.Image.LANCZOS, 'LANCZOS4': cv2.INTER_LANCZOS4,
} }
interp_mode = cfg.SSD.RESIZE.INTERP_MODE interp_mode = cfg.SSD.RESIZE.INTERP_MODE
self._interp_mode = [interp_list[key] for key in interp_mode] self._interp_mode = [interp_list[key] for key in interp_mode]
def resize_image(self, im): def resize_image(self, im):
rand = npr.randint(0, len(self._interp_mode)) rand = npr.randint(0, len(self._interp_mode))
im = PIL.Image.fromarray(im) return cv2.resize(
im = im.resize((self._re_width, self._re_height), self._interp_mode[rand]) im, (self._re_width, self._re_height),
return np.array(im) interpolation=self._interp_mode[rand])
\ No newline at end of file \ No newline at end of file
lib/utils/image.py
...@@ -13,7 +13,6 @@ from __future__ import absolute_import ...@@ -13,7 +13,6 @@ from __future__ import absolute_import
from __future__ import division from __future__ import division
from __future__ import print_function from __future__ import print_function
import sys
import cv2 import cv2
import numpy as np import numpy as np
...@@ -24,11 +23,9 @@ from lib.core.config import cfg ...@@ -24,11 +23,9 @@ from lib.core.config import cfg
def resize_image(im, fx, fy): def resize_image(im, fx, fy):
im_shape = im.shape return cv2.resize(
im = PIL.Image.fromarray(im) im, None, fx=fx, fy=fy,
size = (int(np.ceil(im_shape[1] * fx)), int(np.ceil(im_shape[0] * fy))) interpolation=cv2.INTER_LINEAR)
im = im.resize(size, PIL.Image.BILINEAR)
return np.array(im)
# Faster and robust resizing than OpenCV methods # Faster and robust resizing than OpenCV methods
......
Markdown is supported
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!