[IMPLEMENTED] Chunk Upload Support

Feature Requests Implemented Features
#1

I have…

I’m submitting a…

  • [ ] Regression (a behavior that stopped working in a new release)
  • [ ] Bug report
  • [x] Performance issue
  • [ ] Documentation issue or request

Current behavior

Using the API with self hosted, there does not seem to be a way to chunk the upload for large files. And any network failures warrant starting over the upload instead of picking up where upload left off.

Expected behavior

Is there any plans, or functionality I’m unaware of, for Squidex to handle uploading large media files with chunking? If not, I currently am able to upload large files by chunking using TUS protocol to the same IIS server that is running our self hosted Squidex installation. Is there any way to subsequently “point” Squidex to these media files so they can still be managed in the Squidex admin console?

Minimal reproduction of the problem

Upload a 200MB file to the API using HTTP call.

Environment

App Name:

  • [ ] Self hosted with docker
  • [x] Self hosted with IIS
  • [ ] Self hosted with other version
  • [ ] Cloud version

Version: [VERSION]

Browser:

  • [ x] Chrome (desktop)
  • [ ] Chrome (Android)
  • [ ] Chrome (iOS)
  • [ ] Firefox
  • [ ] Safari (desktop)
  • [ ] Safari (iOS)
  • [ ] IE
  • [ ] Edge

Others:

#2

Hi,

there is no implementation like that available right now. But TUS looks interesting. Just found this: https://github.com/tusdotnet/tusdotnet

#3

Yes. I’m actually using tusdotnet to upload large files to the server. Being I can upload these files to the same server that I’m running Squidex, is there any way to associate the files to Squidex w/out uploading them again via the API?

#4

Technically you could upload a dummy asset and then replace the file, but then some other features like metadata detection and so on do not work anymore. But if you have already uploaded it to the same server you should have far less issues with network connectivity if you create the asset using the API and run your code on this server.

If your assets are already reachable using another API or file server you can just add the URL to your content of course.

But besides that there is no way at the moment.

#5

I had a look to the tus protocol. But it is really complicated. Yes, there is an implementation with tusdotnet, but to make it work with Squidex for all platforms I have to implement at least 4 store providers (Mongo, Azure, S3, GC). Lets see what I can do, but it is much more work than expected.

EDIT: None of the cloud providers I have tested supports appending data.

#6

Actually, now that I deployed the TUS solution and tested it on the server, with self-hosted Squidex installation, it does perform pretty well. I used the TUS separately, and uploading to the Squidex API subsequently only takes a matter of seconds, even for large files. And this way I can still preserve the correct metadata. Seems to not be a problem after all, so please don’t make any changes on my account for this specific issue. However, for a future feature, a chunking option may be nice.

#7

Great, that you solved it. But I think your feedback here is helpful and valuable and brings the product forward. So thanks a lot for that.

#8

:+1: No problem. Hope the feature addition goes well.

#9

Which client do you use for the upload?

#10

I’m using tus-js-client.

1 Like
#11

This tus.io thing is interesting. But very buggy. All implementations I have seen are either broken or incomplete :frowning:

#12

I have added tus support to the API for Assets endpoints.

#13

This topic was automatically closed after 2 days. New replies are no longer allowed.