Water meters #1

anton-br · 2018-01-09T14:15:54Z

No description provided.

…_to_blosc

… in examples and add empty string in .gitattributes

…_blosc

…rt_to_blosc

…data

…s in format_data

GregoryIvanov · 2018-01-17T13:26:30Z

meters/batch/meter_batch.py

+    @action
+    @inbatch_parallel(init='indices', src='images', post='assemble')
+    def normalize_images(self, ind, src='images'):
+        """ Normalize pixel values from (0, 255) to (0, 1).


pixels' values

GregoryIvanov · 2018-01-17T13:27:17Z

meters/batch/meter_batch.py

+from ..dataset.dataset import ImagesBatch, action, inbatch_parallel, any_action_failed
+
+class MeterBatch(ImagesBatch):
+    """Class to create batch with water meter"""


Кажется, лучше написать Water meters' batch class

Причем в ридми указывалось, что класс подойдет для любых счетчиков

GregoryIvanov · 2018-01-17T13:28:03Z

meters/batch/meter_batch.py

+            dataset index
+
+        src : str
+            the name of the placeholder with data


data placeholder's name

GregoryIvanov · 2018-01-17T13:29:22Z

meters/batch/meter_batch.py

+        Parameters
+        ----------
+        ind : str or int
+            dataset index


dataset's index

GregoryIvanov · 2018-01-17T13:32:08Z

meters/batch/meter_batch.py

+        Returns
+        -------
+            array with indices from batch"""
+        _ = args, kwargs


kwargs тут лишние, так как используются дальше. args нигде не использутся (и от класса никто не наследуется), нужны ли они вообще?

лишние kwargs и args удалил везде

GregoryIvanov · 2018-01-17T13:42:54Z

meters/batch/meter_batch.py

+            dataset index
+
+        src : str
+            the name of the placeholder with data


data placeholder's name

GregoryIvanov · 2018-01-17T13:44:02Z

meters/batch/meter_batch.py

+
+    @action
+    @inbatch_parallel(init='_init_component', src='images', dst='bbox', target='threads')
+    def crop_to_bbox(self, ind, *args, src='images', dst='bbox', **kwargs):


*args лучше поставить непосредственно перед kwargs

GregoryIvanov · 2018-01-17T13:45:04Z

meters/batch/meter_batch.py

+
+        dst : str
+            the name of the placeholder in witch the result will be recorded"""
+        _ = args, kwargs


А нужны ли в этой функции args и kwargs?

GregoryIvanov · 2018-01-17T13:47:05Z

meters/batch/meter_batch.py

+        _ = args, kwargs
+        image = self.get(ind, src)
+        coord_str = self.get(ind, 'coordinates')
+        x, y, width, height = [int(val) for val in coord_str.split()]


x, y, width, height = map(int, coord_str.split())

исправил

GregoryIvanov · 2018-01-17T14:04:26Z

meters/batch/meter_batch.py

+        return normalize_image
+
+    def _init_component(self, *args, **kwargs):
+        """Create and preallocate a new attribute with the name ``dst`` if it


Create a new attribute with the name specified by kwargs['dst'] and preallocate memory for it

GregoryIvanov · 2018-01-17T14:04:57Z

meters/batch/meter_batch.py

+
+    def _init_component(self, *args, **kwargs):
+        """Create and preallocate a new attribute with the name ``dst`` if it
+        does not exist and return batch indices


batch's indices

исправил ошибки в английском везде

GregoryIvanov · 2018-01-17T14:06:37Z

meters/batch/meter_batch.py

+    @action
+    @inbatch_parallel(init='_init_component', src='images', dst='bbox', target='threads')
+    def crop_to_bbox(self, ind, *args, src='images', dst='bbox', **kwargs):
+        """Create cropped attr with crop image use ``coordinates``


Crop area from image using coordinates attribute

GregoryIvanov · 2018-01-17T14:06:56Z

meters/batch/meter_batch.py

+        Parameters
+        ----------
+        ind : str or int
+            dataset index


dataset's index

GregoryIvanov · 2018-01-17T14:07:55Z

meters/batch/meter_batch.py

+        _ = args, kwargs
+        i = self.get_pos(None, src, ind)
+        label = getattr(self, src)[i]
+        more_label = np.array([int(i) for i in label.replace(',', '')] + [None])[:-1]


я бы пояснил, зачем нужен None и [:-1]

добавил коммент

dmylzenova · 2018-01-30T10:11:42Z

meters/batch/meter_batch.py

+        src : str
+            data placeholder's name
+        dst : str
+            the name of the placeholder's in witch the result will be recorded


which, а вообще, по-моему, in which можно заменить на where
of the placeholder's - притяжательный падеж ни к чему, если перед этим использован предлог of
а вообще, не очень понятно, что за placeholder?

заменил на components, так правильнее

GregoryIvanov

Check your English.
Check docstrings.
Don't use 'we' in instructions

GregoryIvanov · 2018-01-31T07:59:19Z

README.md

+# About Meters
+
+Meters has two module: [``batch``](https://github.com/analysiscenter/meters/tree/master/meters/batch) and [``pipelines``](https://github.com/analysiscenter/meters/tree/master/meters/pipelines)


two modules

GregoryIvanov · 2018-01-31T08:04:53Z

docs/intro/batch.rst

+
+* ``images``, input images
+* ``labels``, targets for images - array of strings with numbers
+* ``coordinates``, array with four numbers - coordinates of one of the top left corner of the bbox, height and width


-- coordinates of the top-left corner, height and width of the bounding box

GregoryIvanov · 2018-01-31T08:06:00Z

docs/intro/batch.rst

+* ``images``, input images
+* ``labels``, targets for images - array of strings with numbers
+* ``coordinates``, array with four numbers - coordinates of one of the top left corner of the bbox, height and width
+* ``display``, array with images cropped by ``coordinates``


coordinates don't crop anything.

Array with cropped meters' values or smth like that

GregoryIvanov · 2018-01-31T08:06:19Z

docs/intro/batch.rst

+* ``display``, array with images cropped by ``coordinates``
+* ``digits``, array with ``num_split`` numbers from the meter.
+
+Actions of MeterBatch allows to e.g.:


GregoryIvanov · 2018-01-31T08:07:39Z

docs/intro/batch.rst

+Actions of MeterBatch allows to e.g.:
+
+* load images from blosc formats and labels from csv format
+* crop images by coordinates


I would write like this : "crop bounding box's area from an image"

GregoryIvanov · 2018-01-31T08:39:12Z

meters/pipelines/pipelines.py

+        * save prediction to variable named ``prediction``.
+
+        Parameters
+        ----------


same story as for simple_train

GregoryIvanov · 2018-01-31T08:40:21Z

meters/utils/format_data.py

+import numpy as np
+
+def main():
+    """Convert labels and coordinates from csv to normal format. Normal is ambiguous format"""


Very confusing comment

GregoryIvanov · 2018-01-31T08:40:36Z

meters/utils/format_data.py

+        raise ValueError("Missing required argument '-s'")
+
+def format_labels(src):
+    """Convert labels from csv to normat format"""


the same story

GregoryIvanov · 2018-01-31T08:40:48Z

meters/utils/format_data.py

+    return labels
+
+def format_coordinates(src_coord, labels):
+    """Convert coordinates from csv to normat format"""


GregoryIvanov · 2018-01-31T08:40:56Z

meters/utils/format_data.py

+    return data
+
+def format_data(src_data):
+    """Convert data from csv to normat format"""


roman-kh · 2018-01-31T10:33:17Z

meters/__init__.py

+__version__ = '0.1.0'
+
+
+if sys.version_info < (3, 5):


This is not needed, as dataset will raise an exception even earlier

roman-kh · 2018-01-31T10:33:46Z

meters/batch/meter_batch.py

+"""Batch class for water meter task"""
+import numpy as np
+
+from ..dataset.dataset import ImagesBatch, action, inbatch_parallel, any_action_failed, DatasetIndex


one dataset is not enough?

No, pylint can not find imported modules, like ImagesBatch and etc.

roman-kh · 2018-01-31T10:35:34Z

meters/batch/meter_batch.py

+        ----------
+        results : array
+            loaded data
+        Returns


add blank line before

done, i corrected all comments.

…removed from meters/__init__

…e batch class

…f pipelines

anton-br added 21 commits January 9, 2018 17:07

Add initial files and dataset

648928a

Add batch file, example and convert to blosc file

ce4b1b0

Rename water_batch to meter_batch, add README, change argv in convert…

886f62e

…_to_blosc

correction of a typographical error in README, write right batch name…

10bbfcb

… in examples and add empty string in .gitattributes

Update readme, add discription to examples and add dosc to convert_to…

ac5988b

…_blosc

Add docsting and update MeterBatch

15cbc98

Fix spaces in README

dac1b0a

Update README

8048b76

Update docstring

1392616

Update shippable, add docs in meters_batch, fix import order in conve…

262889a

…rt_to_blosc

Update docs and delete extra files

1d7af66

Fix pylint

b08a930

update __init__ and delete extra files

9c97547

Fixed imports in examples

849a5e2

Added normal spaces into index and dell _build

90e3c35

Add file with lead labels and coordinates to one type

6b13b1d

fix spaces in README, docs and utils

eb485c9

Add components into convert_to_blosc and fix index columns in format_…

6ea7950

…data

Added simple pipeline with load and crop pipelines

d1776db

delete dataset as part of the repository

d29222b

Add simple pipelines from pipelines, update meter_batch and fix error…

ca9033e

…s in format_data

anton-br requested review from dmylzenova, GregoryIvanov and roman-kh January 17, 2018 13:15

GregoryIvanov reviewed Jan 17, 2018

View reviewed changes

meters/batch/meter_batch.py Outdated

dataset index

src : str

the name of the placeholder with data

Copy link

GregoryIvanov Jan 17, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

data placeholder's name

GregoryIvanov reviewed Jan 17, 2018

View reviewed changes

meters/batch/meter_batch.py Outdated

Parameters

----------

ind : str or int

dataset index

Copy link

GregoryIvanov Jan 17, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dataset's index

Update batch class in example, change docstrings in pipelines

619ba80

GregoryIvanov reviewed Jan 17, 2018

View reviewed changes

dmylzenova reviewed Jan 30, 2018

View reviewed changes

Update docs in meters_batch and pipeline class

4a712e5

GregoryIvanov suggested changes Jan 31, 2018

View reviewed changes

roman-kh suggested changes Jan 31, 2018

View reviewed changes

anton-br and others added 26 commits January 31, 2018 16:10

Fixed docs in meters_batch, pipelines and format_data, version check …

ce416c4

…removed from meters/__init__

Updated docstring, batch class and pipeline factory

046056a

Fixed utils and added --recursive into index.rst

4f90518

Splinted coordinates on four columns

da34e41

Add bb train and pred

4d21c1f

Add pipelines with train model to predict coordinates of bb and updat…

c32af91

…e batch class

Add full predict pipeline and update batch class

d7e9353

Added pipeline factory example

abbad1b

update pipeline and batch class

679ca9e

Update examples

76fb2ac

Add working version of pipeline factory examples and actual version o…

afe89b6

…f pipelines

add comonents dict to pipelines and update batch class

78c45b1

add docsting into class pipeline factory

4c03ae5

update dataset and add examples of pipeline factory

e5d696d

update docs and added examples without pretrained model

c0edc4f

Update examples and change the preloaded models type to "static"

214a127

par actions return tuples

1a8daf1

Added pipeline that contains all loaded models

190d9d9

Update dataset

230dcb2

deleted prints from batch class

abeb29a

update format_data

1063857

added percpective transform action

2f594d4

update dataset

f97da77

fix pylint

409876c

fixed error with float coordinates of bbox

fca44ec

Deleted exception name in format_data and fixed bug in meter_batch

de17ca1

		# About Meters

		Meters has two module: [``batch``](https://github.com/analysiscenter/meters/tree/master/meters/batch) and [``pipelines``](https://github.com/analysiscenter/meters/tree/master/meters/pipelines)

Water meters #1

Are you sure you want to change the base?

Water meters #1

Conversation

anton-br commented Jan 9, 2018

GregoryIvanov Jan 17, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

GregoryIvanov left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anton-br Jan 31, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

GregoryIvanov Jan 17, 2018 •

edited

Loading

anton-br Jan 31, 2018 •

edited

Loading