Class-typed fields are required when inserting a new document #1851

Initial Checks

• I have read and followed the docs and still think this is a bug

Description

I have this dataclass:

class QuoteFile(BaseDoc):
    quote_file_id: int = None
    extracted_data: Optional[ExtractedDocumentData] = None

When indexing a new document, extracted_data is always required no matter what I do (type it with Optional etc). This is especially caused by BaseDocIndex._get_values_by_column().

I have finally forked docarray, will send a PR for various minor fixes like this one:

You can see my changes here, the highlighted line returns None when the field object is optional and not given for a new document during indexing.

Example Code

No response

Python, DocArray & OS Version

Python 3.8.12
docarray==0.40.0

Affected Components

• Vector Database / Index
• Representing
• Sending
• storing
• multi modal data type