Skip to content

The load_from_folder module

The load_from_folder module contains the classes that are necessary to load data from disk and these are inspired by the ImageFolder class in the torchvision library. This module is designed with one specific case in mind. Such case is the following: given a multi-modal dataset with tabular data, images and text, the images do not fit in memory, and therefore, they have to be loaded from disk. However, as any other functionality in this library, there is some flexibility and some additional cases can also be addressed using this module.

For this module to be used, the datasets must be prepared in a certain way:

  1. the tabular data must contain a column with the images names as stored in disk, including the extension (.jpg, .png, etc...).

  2. Regarding to the text dataset, the tabular data can contain a column with the texts themselves or the names of the files containing the texts as stored in disk.

The tabular data might or might not fit in disk itself. If it does not, please see the ChunkPreprocessor utilities at the preprocessing module and the examples folder in the repo, which illustrate such case. Finally note that only csv format is currently supported in that case (more formats might come soon).

TabFromFolder

This class is used to load tabular data from disk. The current constrains are:

  1. The only file format supported right now is csv
  2. The csv file must contain headers

For examples, please, see the examples folder in the repo.

Parameters:

Name Type Description Default
fname str

the name of the csv file

 required
directory Optional[str]

the path to the directory where the csv file is located. If None, a TabFromFolder reference object must be provided

 None
target_col Optional[str]

the name of the target column. If None, a TabFromFolder reference object must be provided

 None
preprocessor Optional[TabularPreprocessor]

a fitted TabularPreprocessor object. If None, a TabFromFolder reference object must be provided

 None
text_col Optional[Union[str, List[str]]]

the name of the column with the texts themselves or the names of the files that contain the text dataset. If None, either there is no text column or a TabFromFolder reference object must be provided

 None
img_col Optional[Union[str, List[str]]]

the name of the column with the the names of the images. If None, either there is no image column or a TabFromFolder reference object must be provided

 None
ignore_target bool

whether to ignore the target column. This is normally set to True when this class is used for a test dataset.

 False
reference Optional[Any]

a reference TabFromFolder object. If provided, the TabFromFolder object will be created using the attributes of the reference object. This is useful to instantiate a TabFromFolder object for evaluation or test purposes

 None
verbose Optional[int]

verbosity. If 0, no output will be printed during the process.

 1
Source code in pytorch_widedeep/load_from_folder/tabular/tabular_from_folder.py 
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
class TabFromFolder:
    """
    This class is used to load tabular data from disk. The current constrains are:

    1. The only file format supported right now is csv
    2. The csv file must contain headers

    For examples, please, see the examples folder in the repo.

    Parameters
    ----------
    fname: str
        the name of the csv file
    directory: str, Optional, default = None
        the path to the directory where the csv file is located. If None,
        a `TabFromFolder` reference object must be provided
    target_col: str, Optional, default = None
        the name of the target column. If None, a `TabFromFolder` reference
        object must be provided
    preprocessor: `TabularPreprocessor`, Optional, default = None
        a fitted `TabularPreprocessor` object. If None, a `TabFromFolder`
        reference object must be provided
    text_col: str, Optional, default = None
        the name of the column with the texts themselves or the names of the
        files that contain the text dataset. If None, either there is no text
        column or a `TabFromFolder` reference object must be provided
    img_col: str, Optional, default = None
        the name of the column with the the names of the images. If None,
        either there is no image column or a `TabFromFolder` reference object
        must be provided
    ignore_target: bool, default = False
        whether to ignore the target column. This is normally set to True when
        this class is used for a test dataset.
    reference: `TabFromFolder`, Optional, default = None
        a reference `TabFromFolder` object. If provided, the `TabFromFolder`
        object will be created using the attributes of the reference object.
        This is useful to instantiate a `TabFromFolder` object for evaluation
        or test purposes
    verbose: int, default = 1
        verbosity. If 0, no output will be printed during the process.
    """

    def __init__(
        self,
        fname: str,
        directory: Optional[str] = None,
        target_col: Optional[str] = None,
        preprocessor: Optional[TabularPreprocessor] = None,
        text_col: Optional[Union[str, List[str]]] = None,
        img_col: Optional[Union[str, List[str]]] = None,
        ignore_target: bool = False,
        reference: Optional[Any] = None,  # is Type["TabFromFolder"],
        verbose: Optional[int] = 1,
    ):
        self.fname = fname
        self.ignore_target = ignore_target
        self.verbose = verbose

        if reference is not None:
            (
                self.directory,
                self.target_col,
                self.preprocessor,
                self.text_col,
                self.img_col,
            ) = self._set_from_reference(reference, preprocessor)
        else:
            assert (
                directory is not None
                and (target_col is not None and not ignore_target)
                and preprocessor is not None
            ), (
                "if no reference is provided, 'directory', 'target_col' and 'preprocessor' "
                "must be provided"
            )

            self.directory = directory
            self.target_col = target_col
            self.preprocessor = preprocessor
            self.text_col = text_col
            self.img_col = img_col

        assert (
            self.preprocessor.is_fitted
        ), "The preprocessor must be fitted before passing it to this class"

    def get_item(self, idx: int) -> Tuple[  # noqa: C901
        np.ndarray,
        Optional[Union[str, List[str]]],
        Optional[Union[str, List[str]]],
        Optional[Union[int, float]],
    ]:
        """
        This method is used to retrieve a sample from the csv file

        Parameters
        ----------
        idx: int
            the index of the sample to retrieve

        Returns
        -------
        Tuple
            a tuple with the processed tabular data, the text data and/or the
            image data, and the target variable
        """

        path = os.path.join(self.directory, self.fname)

        try:
            if not hasattr(self, "colnames"):
                self.colnames = pd.read_csv(path, nrows=0).columns.tolist()

            # TO DO: we need to look into this as the treatment is different
            # whether the csv contains headers or not. For the time being we
            # will require that the csv file has headers

            _sample = pd.read_csv(
                path, skiprows=lambda x: x != idx + 1, header=None
            ).values
            sample = pd.DataFrame(_sample, columns=self.colnames)
        except Exception:
            raise ValueError("Currently only csv format is supported.")

        text_fnames_or_text: Optional[Union[str, List[str]]] = None
        if self.text_col is not None:
            if isinstance(self.text_col, list):
                text_fnames_or_text = [
                    sample[col].to_list()[0] for col in self.text_col
                ]
            else:
                text_fnames_or_text = sample[self.text_col].to_list()[0]

        img_fname: Optional[Union[str, List[str]]] = None
        if self.img_col is not None:
            if isinstance(self.img_col, list):
                img_fname = [sample[col].to_list()[0] for col in self.img_col]
            else:
                img_fname = sample[self.img_col].to_list()[0]

        processed_sample = self.preprocessor.transform_sample(sample)

        if not self.ignore_target:
            target = sample[self.target_col].to_list()[0]
        else:
            target = None

        return processed_sample, text_fnames_or_text, img_fname, target

    def _set_from_reference(
        self,
        reference: Any,  # is Type["TabFromFolder"],
        preprocessor: Optional[TabularPreprocessor],
    ) -> Tuple[
        str,
        str,
        TabularPreprocessor,
        Optional[Union[str, List[str]]],
        Optional[Union[str, List[str]]],
    ]:
        (
            directory,
            target_col,
            _preprocessor,
            text_col,
            img_col,
        ) = self._get_from_reference(reference)

        if preprocessor is not None:
            preprocessor = preprocessor
            if self.verbose:
                UserWarning(
                    "The preprocessor from the reference object is overwritten "
                    "by the provided preprocessor"
                )
        else:
            preprocessor = _preprocessor

        return directory, target_col, preprocessor, text_col, img_col

    @staticmethod
    def _get_from_reference(
        reference: Any,  # is Type["TabFromFolder"],
    ) -> Tuple[str, str, TabularPreprocessor, Optional[str], Optional[str]]:
        return (
            reference.directory,
            reference.target_col,
            reference.preprocessor,
            reference.text_col,
            reference.img_col,
        )

    def __repr__(self) -> str:  # noqa: C901
        list_of_params: List[str] = []
        if self.fname is not None:
            list_of_params.append("fname={fname}")
        if self.directory is not None:
            list_of_params.append("directory={directory}")
        if self.target_col is not None:
            list_of_params.append("target_col={target_col}")
        if self.preprocessor is not None:
            list_of_params.append(
                f"preprocessor={self.preprocessor.__class__.__name__}"
            )
        if self.text_col is not None:
            if isinstance(self.text_col, list):
                list_of_params.append(
                    f"text_col={[text_col for text_col in self.text_col]}"
                )
            else:
                list_of_params.append("text_col={text_col}")
        if self.img_col is not None:
            if isinstance(self.img_col, list):
                list_of_params.append(
                    f"img_col={[img_col for img_col in self.img_col]}"
                )
            else:
                list_of_params.append("img_col={img_col}")
        if self.ignore_target is not None:
            list_of_params.append("ignore_target={ignore_target}")
        if self.verbose is not None:
            list_of_params.append("verbose={verbose}")
        all_params = ", ".join(list_of_params)
        return f"{self.__class__.__name__}({all_params.format(**self.__dict__)})"

get_item 

get_item(idx)

This method is used to retrieve a sample from the csv file

Parameters:

Name Type Description Default
idx int

the index of the sample to retrieve

 required

Returns:

Type Description
Tuple

a tuple with the processed tabular data, the text data and/or the image data, and the target variable
Source code in pytorch_widedeep/load_from_folder/tabular/tabular_from_folder.py 
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
def get_item(self, idx: int) -> Tuple[  # noqa: C901
    np.ndarray,
    Optional[Union[str, List[str]]],
    Optional[Union[str, List[str]]],
    Optional[Union[int, float]],
]:
    """
    This method is used to retrieve a sample from the csv file

    Parameters
    ----------
    idx: int
        the index of the sample to retrieve

    Returns
    -------
    Tuple
        a tuple with the processed tabular data, the text data and/or the
        image data, and the target variable
    """

    path = os.path.join(self.directory, self.fname)

    try:
        if not hasattr(self, "colnames"):
            self.colnames = pd.read_csv(path, nrows=0).columns.tolist()

        # TO DO: we need to look into this as the treatment is different
        # whether the csv contains headers or not. For the time being we
        # will require that the csv file has headers

        _sample = pd.read_csv(
            path, skiprows=lambda x: x != idx + 1, header=None
        ).values
        sample = pd.DataFrame(_sample, columns=self.colnames)
    except Exception:
        raise ValueError("Currently only csv format is supported.")

    text_fnames_or_text: Optional[Union[str, List[str]]] = None
    if self.text_col is not None:
        if isinstance(self.text_col, list):
            text_fnames_or_text = [
                sample[col].to_list()[0] for col in self.text_col
            ]
        else:
            text_fnames_or_text = sample[self.text_col].to_list()[0]

    img_fname: Optional[Union[str, List[str]]] = None
    if self.img_col is not None:
        if isinstance(self.img_col, list):
            img_fname = [sample[col].to_list()[0] for col in self.img_col]
        else:
            img_fname = sample[self.img_col].to_list()[0]

    processed_sample = self.preprocessor.transform_sample(sample)

    if not self.ignore_target:
        target = sample[self.target_col].to_list()[0]
    else:
        target = None

    return processed_sample, text_fnames_or_text, img_fname, target

WideFromFolder

Bases: TabFromFolder

This class is mostly identical to TabFromFolder but exists because we want to separate the treatment of the wide and the deep tabular components

Parameters:

Name Type Description Default
fname str

the name of the csv file

 required
directory Optional[str]

the path to the directory where the csv file is located. If None, a WideFromFolder reference object must be provided

 None
target_col Optional[str]

the name of the target column. If None, a WideFromFolder reference object must be provided

 None
preprocessor Optional[TabularPreprocessor]

a fitted TabularPreprocessor object. If None, a WideFromFolder reference object must be provided

 None
text_col Optional[str]

the name of the column with the texts themselves or the names of the files that contain the text dataset. If None, either there is no text column or a WideFromFolder reference object must be provided=

 None
img_col Optional[str]

the name of the column with the the names of the images. If None, either there is no image column or a WideFromFolder reference object must be provided

 None
ignore_target bool

whether to ignore the target column. This is normally used when this class is used for a test dataset.

 False
reference Optional[Any]

a reference WideFromFolder object. If provided, the WideFromFolder object will be created using the attributes of the reference object. This is useful to instantiate a WideFromFolder object for evaluation or test purposes

 None
verbose int

verbosity. If 0, no output will be printed during the process.

 1
Source code in pytorch_widedeep/load_from_folder/tabular/tabular_from_folder.py 
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
class WideFromFolder(TabFromFolder):
    """
    This class is mostly identical to `TabFromFolder` but exists because we
    want to separate the treatment of the wide and the deep tabular
    components

    Parameters
    ----------
    fname: str
        the name of the csv file
    directory: str, Optional, default = None
        the path to the directory where the csv file is located. If None,
        a `WideFromFolder` reference object must be provided
    target_col: str, Optional, default = None
        the name of the target column. If None, a `WideFromFolder` reference
        object must be provided
    preprocessor: `TabularPreprocessor`, Optional, default = None
        a fitted `TabularPreprocessor` object. If None, a `WideFromFolder`
        reference object must be provided
    text_col: str, Optional, default = None
        the name of the column with the texts themselves or the names of the
        files that contain the text dataset. If None, either there is no text
        column or a `WideFromFolder` reference object must be provided=
    img_col: str, Optional, default = None
        the name of the column with the the names of the images. If None,
        either there is no image column or a `WideFromFolder` reference object
        must be provided
    ignore_target: bool, default = False
        whether to ignore the target column. This is normally used when this
        class is used for a test dataset.
    reference: `WideFromFolder`, Optional, default = None
        a reference `WideFromFolder` object. If provided, the `WideFromFolder`
        object will be created using the attributes of the reference object.
        This is useful to instantiate a `WideFromFolder` object for evaluation
        or test purposes
    verbose: int, default = 1
        verbosity. If 0, no output will be printed during the process.
    """

    def __init__(
        self,
        fname: str,
        directory: Optional[str] = None,
        target_col: Optional[str] = None,
        preprocessor: Optional[TabularPreprocessor] = None,
        text_col: Optional[str] = None,
        img_col: Optional[str] = None,
        ignore_target: bool = False,
        reference: Optional[Any] = None,  # is Type["WideFromFolder"],
        verbose: int = 1,
    ):
        super(WideFromFolder, self).__init__(
            fname=fname,
            directory=directory,
            target_col=target_col,
            preprocessor=preprocessor,
            text_col=text_col,
            img_col=img_col,
            reference=reference,
            ignore_target=ignore_target,
            verbose=verbose,
        )

TextFromFolder

This class is used to load the text dataset (i.e. the text files) from a folder, or to retrieve the text given a texts column specified within the preprocessor object.

For examples, please, see the examples folder in the repo.

Parameters:

Name Type Description Default
preprocessor Union[TextPreprocessor, ChunkTextPreprocessor, HFPreprocessor, ChunkHFPreprocessor, List[TextPreprocessor], List[ChunkTextPreprocessor], List[HFPreprocessor], List[ChunkHFPreprocessor]]

The preprocessor used to process the text. It must be fitted before using this class

 required
Source code in pytorch_widedeep/load_from_folder/text/text_from_folder.py 
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
class TextFromFolder:
    """
    This class is used to load the text dataset (i.e. the text files) from a
    folder, or to retrieve the text given a texts column specified within the
    preprocessor object.

    For examples, please, see the examples folder in the repo.

    Parameters
    ----------
    preprocessor:
        The preprocessor used to process the text. It must be fitted before using
        this class
    """

    def __init__(
        self,
        preprocessor: Union[
            TextPreprocessor,
            ChunkTextPreprocessor,
            HFPreprocessor,
            ChunkHFPreprocessor,
            List[TextPreprocessor],
            List[ChunkTextPreprocessor],
            List[HFPreprocessor],
            List[ChunkHFPreprocessor],
        ],
    ):
        if isinstance(preprocessor, list):
            for p in preprocessor:
                assert (
                    p.is_fitted
                ), "All preprocessors must be fitted before using this class"
        else:
            assert (
                preprocessor.is_fitted
            ), "The preprocessor must be fitted before using this class"

        self.preprocessor = preprocessor

    def get_item(
        self, text: Union[str, List[str]]
    ) -> Union[np.ndarray, List[np.ndarray]]:
        """
        Given a text or a list of texts corresponding to different text
        columns, this method will return the processed text or a list of
        processed texts

        Parameters
        ----------
        text: Union[str, List[str]]
            The text or list of texts to be processed

        Returns
        -------
        Union[np.ndarray, List[np.ndarray]]
            The processed text or a list of processed texts
        """
        if isinstance(self.preprocessor, list):
            assert isinstance(text, list)
            processed_sample: Union[np.ndarray, List[np.ndarray]] = [
                self._preprocess_one_sample(t, self.preprocessor[i])
                for i, t in enumerate(text)
            ]
        else:
            assert isinstance(text, str)
            processed_sample = self._preprocess_one_sample(text, self.preprocessor)

        return processed_sample

    def _preprocess_one_sample(
        self,
        text: str,
        preprocessor: Union[
            TextPreprocessor,
            ChunkTextPreprocessor,
            HFPreprocessor,
            ChunkHFPreprocessor,
        ],
    ) -> np.ndarray:
        if (
            isinstance(preprocessor, ChunkTextPreprocessor)
            and preprocessor.root_dir is not None
        ):
            path = os.path.join(preprocessor.root_dir, text)

            with open(path, "r") as f:
                sample = f.read().replace("\n", "")
        else:
            sample = text

        processed_sample = preprocessor.transform_sample(sample)

        return processed_sample

    def __repr__(self):
        if isinstance(self.preprocessor, list):
            return f"{self.__class__.__name__}({[p.__class__.__name__ for p in self.preprocessor]})"
        else:
            return f"{self.__class__.__name__}({self.preprocessor.__class__.__name__})"

get_item 

get_item(text)

Given a text or a list of texts corresponding to different text columns, this method will return the processed text or a list of processed texts

Parameters:

Name Type Description Default
text Union[str, List[str]]

The text or list of texts to be processed

 required

Returns:

Type Description
Union[ndarray, List[ndarray]]

The processed text or a list of processed texts
Source code in pytorch_widedeep/load_from_folder/text/text_from_folder.py 
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
def get_item(
    self, text: Union[str, List[str]]
) -> Union[np.ndarray, List[np.ndarray]]:
    """
    Given a text or a list of texts corresponding to different text
    columns, this method will return the processed text or a list of
    processed texts

    Parameters
    ----------
    text: Union[str, List[str]]
        The text or list of texts to be processed

    Returns
    -------
    Union[np.ndarray, List[np.ndarray]]
        The processed text or a list of processed texts
    """
    if isinstance(self.preprocessor, list):
        assert isinstance(text, list)
        processed_sample: Union[np.ndarray, List[np.ndarray]] = [
            self._preprocess_one_sample(t, self.preprocessor[i])
            for i, t in enumerate(text)
        ]
    else:
        assert isinstance(text, str)
        processed_sample = self._preprocess_one_sample(text, self.preprocessor)

    return processed_sample

ImageFromFolder

This class is used to load the image dataset from disk. It is inspired by the ImageFolder class at the torchvision library. Here, we have simply adapted to work within the context of a Wide and Deep multi-modal model.

For examples, please, see the examples folder in the repo.

Parameters:

Name Type Description Default
directory Optional[Union[str, List[str]]]

the path to the directory where the images are located. If None, a preprocessor must be provided.

 None
preprocessor Optional[Union[ImagePreprocessor, List[ImagePreprocessor]]]

a fitted ImagePreprocessor object.

 None
loader Callable[[str], Any]

a function to load a sample given its path.

 default_loader
extensions Optional[Tuple[str, ...]]

a tuple with the allowed extensions. If None, IMG_EXTENSIONS will be used where IMG_EXTENSIONS =".jpg", ".jpeg", ".png", ".ppm", ".bmp", ".pgm", ".tif", ".tiff", ".webp"

 None
transforms Optional[Any]

a torchvision.transforms object. If None, this class will simply return an array representation of the PIL Image

 None
Source code in pytorch_widedeep/load_from_folder/image/image_from_folder.py 
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
class ImageFromFolder:
    """
    This class is used to load the image dataset from disk. It is inspired by
    the `ImageFolder` class at the `torchvision` library. Here, we have
    simply adapted to work within the context of a Wide and Deep multi-modal
    model.

    For examples, please, see the examples folder in the repo.

    Parameters
    ----------
    directory: str, Optional, default = None
        the path to the directory where the images are located. If None, a
        preprocessor must be provided.
    preprocessor: `ImagePreprocessor`, Optional, default = None
        a fitted `ImagePreprocessor` object.
    loader: Callable[[str], Any], Optional, default = default_loader
        a function to load a sample given its path.
    extensions: Tuple[str, ...], Optional, default = IMG_EXTENSIONS
        a tuple with the allowed extensions. If None, IMG_EXTENSIONS will be
        used where IMG_EXTENSIONS
        =".jpg", ".jpeg", ".png", ".ppm", ".bmp", ".pgm", ".tif", ".tiff", ".webp"
    transforms: Optional[Any], default = None
        a `torchvision.transforms` object. If None, this class will simply
        return an array representation of the PIL Image
    """

    def __init__(
        self,
        directory: Optional[Union[str, List[str]]] = None,
        preprocessor: Optional[
            Union[ImagePreprocessor, List[ImagePreprocessor]]
        ] = None,
        loader: Callable[[str], Any] = default_loader,
        extensions: Optional[Tuple[str, ...]] = None,
        transforms: Optional[Any] = None,
    ) -> None:
        assert (
            directory is not None or preprocessor is not None
        ), "Either a directory or an instance of ImagePreprocessor(s) must be provided"

        if directory is not None and preprocessor is not None:  # pragma: no cover
            error_msg = (
                "If both 'directory' and 'preprocessor' are provided, the 'img_path' "
                "attribute of the 'preprocessor' must be the same as the 'directory'"
            )
            if isinstance(directory, list):
                assert isinstance(preprocessor, list)
                assert len(directory) == len(preprocessor)
                for d, p in zip(directory, preprocessor):
                    assert d == p.img_path, error_msg
            else:
                assert isinstance(preprocessor, ImagePreprocessor)
                assert directory == preprocessor.img_path, error_msg

        if directory is not None:
            self.directory = directory
        else:
            assert (
                preprocessor is not None
            ), "Either a directory or an instance of ImagePreprocessor must be provided"
            if isinstance(preprocessor, list):
                self.directory = [p.img_path for p in preprocessor]
            else:
                self.directory = preprocessor.img_path

        self.preprocessor = preprocessor
        self.loader = loader
        self.extensions = extensions if extensions is not None else IMG_EXTENSIONS
        self.transforms = transforms
        if self.transforms:
            self.transforms_names = [
                tr.__class__.__name__ for tr in self.transforms.transforms
            ]
        else:
            self.transforms_names = []

            self.transpose = True

    def get_item(
        self, fname: Union[str, List[str]]
    ) -> Union[np.ndarray, List[np.ndarray]]:
        """
        This method is used to load the image dataset(s) from disk.

        Parameters
        ----------
        fname: Union[str, List[str]]
            the name of the image file(s) to load. If a list is provided, the
            method will return a list of numpy arrays. Each element in the list
            corresponds to the image in the different image columns.

        Returns
        -------
        Union[np.ndarray, List[np.ndarray]]
            a numpy array or a list of numpy arrays representing the image(s)
        """

        if isinstance(fname, list):
            if not isinstance(self.directory, list):
                _directory = [self.directory] * len(fname)
            else:
                _directory = self.directory
            if self.preprocessor is not None:
                assert isinstance(self.preprocessor, list)
                processed_sample: Union[np.ndarray, List[np.ndarray]] = [
                    self._preprocess_one_sample(f, d, p)
                    for f, d, p in zip(fname, _directory, self.preprocessor)
                ]
            else:
                processed_sample = [
                    self._preprocess_one_sample(f, d) for f, d in zip(fname, _directory)
                ]
        else:
            assert isinstance(self.directory, str)
            if self.preprocessor is not None:
                assert isinstance(self.preprocessor, ImagePreprocessor)
                processed_sample = self._preprocess_one_sample(
                    fname, self.directory, self.preprocessor
                )
            else:
                processed_sample = self._preprocess_one_sample(fname, self.directory)

        return processed_sample

    def _preprocess_one_sample(
        self,
        fname: str,
        directory: str,
        preprocessor: Optional[ImagePreprocessor] = None,
    ) -> np.ndarray:
        assert has_file_allowed_extension(fname, self.extensions)

        path = os.path.join(directory, fname)
        sample = self.loader(path)

        assert isinstance(sample, (Image.Image, np.ndarray)), (  # pragma: no cover
            "The loader must return an instance of PIL.Image or np.ndarray, "
            f"got {type(sample)} instead"
        )

        if preprocessor is not None:
            if not isinstance(sample, np.ndarray):
                processed_sample = preprocessor.transform_sample(np.asarray(sample))
            else:
                processed_sample = preprocessor.transform_sample(sample)
        else:
            processed_sample = sample  # type: ignore

        prepared_sample = self._prepare_sample(processed_sample)

        return prepared_sample

    def _prepare_sample(  # noqa: C901
        self, processed_sample: Union[np.ndarray, Image.Image]
    ) -> np.ndarray:
        # if an image dataset is used, make sure is in the right format to
        # be ingested by the conv layers

        if isinstance(processed_sample, Image.Image):
            processed_sample = np.asarray(processed_sample)

        # if int must be uint8
        if "int" in str(processed_sample.dtype) and "uint8" != str(
            processed_sample.dtype
        ):
            processed_sample = processed_sample.astype("uint8")

        # if float must be float32
        if "float" in str(processed_sample.dtype) and "float32" != str(
            processed_sample.dtype
        ):
            processed_sample = processed_sample.astype("float32")

        # if there are no transforms, or these do not include ToTensor()
        # (weird or unexpected case, not sure is even possible) then we need
        # to  replicate what ToTensor() does -> transpose axis and normalize if
        # necessary
        if not self.transforms or "ToTensor" not in self.transforms_names:
            if processed_sample.ndim == 2:
                processed_sample = processed_sample[:, :, None]

            processed_sample = processed_sample.transpose(2, 0, 1)

            if "int" in str(processed_sample.dtype):
                processed_sample = (processed_sample / processed_sample.max()).astype(
                    "float32"
                )
        elif "ToTensor" in self.transforms_names:
            # if ToTensor() is included, simply apply transforms
            assert self.transforms_names[0] == "ToTensor", (
                "If ToTensor() is included in the transforms, it must be the "
                "first transform in the list"
            )
            processed_sample = self.transforms(processed_sample)
        else:
            # else apply transforms on the result of calling torch.tensor on
            # processed_sample after all the previous manipulation
            processed_sample = self.transforms(torch.tensor(processed_sample))

        return processed_sample  # type: ignore

    def __repr__(self) -> str:
        list_of_params: List[str] = []
        if self.directory is not None:
            list_of_params.append("directory={directory}")
        if self.preprocessor is not None:
            list_of_params.append(
                f"preprocessor={self.preprocessor.__class__.__name__}"
            )
        if self.loader is not None:
            list_of_params.append(f"loader={self.loader.__name__}")
        if self.extensions is not None:
            list_of_params.append("extensions={extensions}")
        if self.transforms is not None:
            list_of_params.append(f"transforms={self.transforms_names}")
        all_params = ", ".join(list_of_params)
        return f"ImageFromFolder({all_params.format(**self.__dict__)})"

get_item 

get_item(fname)

This method is used to load the image dataset(s) from disk.

Parameters:

Name Type Description Default
fname Union[str, List[str]]

the name of the image file(s) to load. If a list is provided, the method will return a list of numpy arrays. Each element in the list corresponds to the image in the different image columns.

 required

Returns:

Type Description
Union[ndarray, List[ndarray]]

a numpy array or a list of numpy arrays representing the image(s)
Source code in pytorch_widedeep/load_from_folder/image/image_from_folder.py 
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
def get_item(
    self, fname: Union[str, List[str]]
) -> Union[np.ndarray, List[np.ndarray]]:
    """
    This method is used to load the image dataset(s) from disk.

    Parameters
    ----------
    fname: Union[str, List[str]]
        the name of the image file(s) to load. If a list is provided, the
        method will return a list of numpy arrays. Each element in the list
        corresponds to the image in the different image columns.

    Returns
    -------
    Union[np.ndarray, List[np.ndarray]]
        a numpy array or a list of numpy arrays representing the image(s)
    """

    if isinstance(fname, list):
        if not isinstance(self.directory, list):
            _directory = [self.directory] * len(fname)
        else:
            _directory = self.directory
        if self.preprocessor is not None:
            assert isinstance(self.preprocessor, list)
            processed_sample: Union[np.ndarray, List[np.ndarray]] = [
                self._preprocess_one_sample(f, d, p)
                for f, d, p in zip(fname, _directory, self.preprocessor)
            ]
        else:
            processed_sample = [
                self._preprocess_one_sample(f, d) for f, d in zip(fname, _directory)
            ]
    else:
        assert isinstance(self.directory, str)
        if self.preprocessor is not None:
            assert isinstance(self.preprocessor, ImagePreprocessor)
            processed_sample = self._preprocess_one_sample(
                fname, self.directory, self.preprocessor
            )
        else:
            processed_sample = self._preprocess_one_sample(fname, self.directory)

    return processed_sample

WideDeepDatasetFromFolder

Bases: Dataset

This class is the Dataset counterpart of the WideDeepDataset class.

Given a reference tabular dataset, with columns that indicate the path to the images and to the text files or the texts themselves, it will use the [...]FromFolder classes to load the data consistently from disk per batch.

For examples, please, see the examples folder in the repo.

Parameters:

Name Type Description Default
n_samples int

Number of samples in the dataset

 required
tab_from_folder Optional[TabFromFolder]

Instance of the TabFromFolder class

 None
wide_from_folder Optional[WideFromFolder]

Instance of the WideFromFolder class

 None
text_from_folder Optional[TextFromFolder]

Instance of the TextFromFolder class

 None
img_from_folder Optional[ImageFromFolder]

Instance of the ImageFromFolder class

 None
reference Optional[Any]

If reference not None, the text_from_folder and img_from_folder objects will be retrieved from the reference class. This is useful when we want to use a WideDeepDatasetFromFolder class used for a train dataset as a reference for the validation and test datasets. In this case, the text_from_folder and img_from_folder objects will be the same for all three datasets, so there is no need to create a new instance for each dataset.

 None
Source code in pytorch_widedeep/load_from_folder/wd_dataset_from_folder.py 
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
class WideDeepDatasetFromFolder(Dataset):
    """
    This class is the Dataset counterpart of the `WideDeepDataset` class.

    Given a reference tabular dataset, with columns that indicate the path to
    the images and to the text files or the texts themselves, it will use the
    `[...]FromFolder` classes to load the data consistently from disk per batch.

    For examples, please, see the examples folder in the repo.

    Parameters
    ----------
    n_samples: int
        Number of samples in the dataset
    tab_from_folder: TabFromFolder
        Instance of the `TabFromFolder` class
    wide_from_folder: Optional[WideFromFolder], default = None
        Instance of the `WideFromFolder` class
    text_from_folder: Optional[TextFromFolder], default = None
        Instance of the `TextFromFolder` class
    img_from_folder: Optional[ImageFromFolder], default = None
        Instance of the `ImageFromFolder` class
    reference: Type["WideDeepDatasetFromFolder"], default = None
        If `reference` not None, the `text_from_folder` and `img_from_folder`
        objects will be retrieved from the reference class. This is useful
        when we want to use a `WideDeepDatasetFromFolder` class used for a
        train dataset as a reference for the validation and test datasets. In
        this case, the `text_from_folder` and `img_from_folder` objects will
        be the same for all three datasets, so there is no need to create a
        new instance for each dataset.
    """

    def __init__(
        self,
        n_samples: int,
        tab_from_folder: Optional[TabFromFolder] = None,
        wide_from_folder: Optional[WideFromFolder] = None,
        text_from_folder: Optional[TextFromFolder] = None,
        img_from_folder: Optional[ImageFromFolder] = None,
        reference: Optional[Any] = None,  # is Type["WideDeepDatasetFromFolder"],
    ):
        super(WideDeepDatasetFromFolder, self).__init__()

        if tab_from_folder is None and wide_from_folder is None:
            raise ValueError(
                "Either 'tab_from_folder' or 'wide_from_folder' must be not None"
            )

        if reference is not None:
            assert (
                img_from_folder is None and text_from_folder is None
            ), "If reference is not None, 'img_from_folder' and 'text_from_folder' left as None"
            self.text_from_folder, self.img_from_folder = self._get_from_reference(
                reference
            )
        else:
            assert (
                text_from_folder is not None and img_from_folder is not None
            ), "If reference is None, 'img_from_folder' and 'text_from_folder' must be not None"
            self.text_from_folder = text_from_folder
            self.img_from_folder = img_from_folder

        self.n_samples = n_samples
        self.tab_from_folder = tab_from_folder
        self.wide_from_folder = wide_from_folder

    def __getitem__(self, idx: int):  # noqa: C901
        x = (
            Bunch()
        )  # for consistency with WideDeepDataset, but this is just a Dict[str, Any]

        if self.tab_from_folder is not None:
            X_tab, text_fname_or_text, img_fname, y = self.tab_from_folder.get_item(
                idx=idx
            )
            x.deeptabular = X_tab

        if self.wide_from_folder is not None:
            if self.tab_from_folder is None:
                (
                    X_wide,
                    text_fname_or_text,
                    img_fname,
                    y,
                ) = self.wide_from_folder.get_item(idx=idx)
            else:
                X_wide, _, _, _ = self.wide_from_folder.get_item(idx=idx)
            x.wide = X_wide

        if text_fname_or_text is not None:
            # These assertions should never be raised, but just in case...
            assert (
                self.text_from_folder is not None
            ), "text_fname_or_text is not None but self.text_from_folder is None"
            X_text = self.text_from_folder.get_item(text_fname_or_text)
            x.deeptext = X_text

        if img_fname is not None:
            assert (
                self.img_from_folder is not None
            ), "img_fname is not None but self.img_from_folder is None"
            X_img = self.img_from_folder.get_item(img_fname)
            x.deepimage = X_img

        # We are aware that returning sometimes X and sometimes X, y is not
        # the best practice, but is the easiest way at this stage
        if y is not None:
            return x, y
        else:
            return x

    def __len__(self):
        return self.n_samples

    @staticmethod
    def _get_from_reference(
        reference: Type["WideDeepDatasetFromFolder"],
    ) -> Tuple[Optional[TextFromFolder], Optional[ImageFromFolder]]:
        return reference.text_from_folder, reference.img_from_folder

    def __repr__(self) -> str:
        list_of_params: List[str] = []
        list_of_params.append("n_samples={n_samples}")
        if self.tab_from_folder is not None:
            list_of_params.append(
                f"tab_from_folder={self.tab_from_folder.__class__.__name__}"
            )
        if self.wide_from_folder is not None:
            list_of_params.append(
                f"wide_from_folder={self.wide_from_folder.__class__.__name__}"
            )
        if self.text_from_folder is not None:
            list_of_params.append(
                f"text_from_folder={self.text_from_folder.__class__.__name__}"
            )
        if self.img_from_folder is not None:
            list_of_params.append(
                f"img_from_folder={self.img_from_folder.__class__.__name__}"
            )
        all_params = ", ".join(list_of_params)
        return f"WideDeepDatasetFromFolder({all_params.format(**self.__dict__)})"