[Herddb-dev] Herddb clients with go and python

Sun Jan 19 21:23:51 CET 2020

Hello,

Thanks for starting the discussion.
My plan today is to study in detail the current implementation of the
protocol we have in herddb and then make decision whether to go grpc route
or not for the Go language. I will update the team shortly on my findings.

-Amit

On Fri, Jan 17, 2020 at 1:47 AM <herddb-dev-request at lists.herddb.org> wrote:

> Send herddb-dev mailing list submissions to
>         herddb-dev at lists.herddb.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         http://lists.herddb.org/mailman/listinfo/herddb-dev
> or, via email, send a message with subject or body 'help' to
>         herddb-dev-request at lists.herddb.org
>
> You can reach the person managing the list at
>         herddb-dev-owner at lists.herddb.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of herddb-dev digest..."
>
>
> Today's Topics:
>
>    1. Re: R: Herddb clients with go and python (Enrico Olivelli)
>    2. R:  R: Herddb clients with go and python
>       (Alessandro Luccaroni - Diennea)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Fri, 17 Jan 2020 09:09:33 +0100
> From: Enrico Olivelli <eolivelli at gmail.com>
> To: Herddb developers <herddb-dev at lists.herddb.org>
> Subject: Re: [Herddb-dev] R: Herddb clients with go and python
> Message-ID:
>         <CACcefgdSiWB4D=
> B9HY29vmeUEQe3f7fmfnX0YDkWD-kiXu_Xmg at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Alessandro,
>
> Il giorno ven 17 gen 2020 alle ore 09:04 Alessandro Luccaroni - Diennea <
> alessandro.luccaroni at diennea.com> ha scritto:
>
> > We don?t have (as far as I know) many ?expert? in languages outside Java
> > so I think going down the route of supporting protobuf/grpc is a good
> > investment: if we get more traction it?s way easier for a newcomer to
> > provide and support a new client this way.
> >
> >
> >
> > @Enrico: when you talk about ?performing the Java part? do you mean to
> > switch also the Java client to grpc? If the performance are
> good/acceptable
> > I think we should also evaluate to drop the custom binary protocol to
> > streamline the code (this should be feasible right now since we have only
> > one client implementation and we are still in the growing phase of the
> > project)
> >
>
> No, I mean to do the plumbing of the gRPC server into the server.
> Current "custom" protocol is super optimized in order to save as many in in
> memory copies as possible, both in the client and in the server,
> when the server receives a request it does need to create intermediate java
> objects from the parts of the request, it reads directly from the ByteBuf,
> the same is for the client.
> For a resultset the client receives a "big" ByteBuf and it reads the values
> directly from that buffer.
> Any other "higher level" abstract needs to create intermediate Java objects
> or add memory copies.
>
> So by now I don't like to drop current wire protocol.
> Maybe I am just biased by the experience with Flatbuffers we had in 0.7.0.
>
> Enrico
>
>
> >
> >
> > Ale
> >
> >
> >
> > *Da:* herddb-dev-bounces at lists.herddb.org <
> > herddb-dev-bounces at lists.herddb.org> *Per conto di *Enrico Olivelli
> > *Inviato:* gioved? 16 gennaio 2020 22:02
> > *A:* Herddb developers <herddb-dev at lists.herddb.org>
> > *Oggetto:* Re: [Herddb-dev] Herddb clients with go and python
> >
> >
> >
> > Amit
> >
> > Il gio 16 gen 2020, 19:30 Amit Chavan <achavan1 at gmail.com> ha scritto:
> >
> > Hello,
> >
> >
> >
> > What does the team think about writing some clients for herddb in other
> > languages?
> >
> > We can start with Go or python to begin with? Let me know your thoughts?
> >
> >
> >
> > It would be great.
> >
> >
> >
> > For network transport do we use anything specific with the current java
> > implementation?
> >
> >
> >
> > We have a custom protocol, I think you already touched it.
> >
> > If we can support our native binary protocol it would be easier to get
> the
> > same performances as for the java binding.
> >
> >
> >
> > We cam also add some alternative protocol using protobuf or even grpc.
> >
> > Maybe with grcp most of the wire protocol work is already done by build
> > time plugins.
> >
> >
> >
> > If we want to go down the protobuf/grcp way I am happy to do the Java
> > part. I know that Diego knows well grcp so maybe he can work on that
> topic
> > as well (can't talk for me btw)
> >
> >
> >
> > I will start documenting the discussion and design approaches as we start
> > having a discussion on this email thread.
> >
> >
> >
> >  Very good
> >
> >
> >
> >
> >
> > Enrico
> >
> >
> >
> > -Amit
> >
> > _______________________________________________
> > herddb-dev mailing list
> > herddb-dev at lists.herddb.org
> > http://lists.herddb.org/mailman/listinfo/herddb-dev
> >
> >
> > ------------------------------
> >
> > CONFIDENTIALITY & PRIVACY NOTICE
> > This e-mail (including any attachments) is strictly confidential and may
> > also contain privileged information. If you are not the intended
> recipient
> > you are not authorised to read, print, save, process or disclose this
> > message. If you have received this message by mistake, please inform the
> > sender immediately and destroy this e-mail, its attachments and any
> copies.
> > Any use, distribution, reproduction or disclosure by any person other
> than
> > the intended recipient is strictly prohibited and the person responsible
> > may incur in penalties.
> > The use of this e-mail is only for professional purposes; there is no
> > guarantee that the correspondence towards this e-mail will be read only
> by
> > the recipient, because, under certain circumstances, there may be a need
> to
> > access this email by third subjects belonging to the Company.
> > _______________________________________________
> > herddb-dev mailing list
> > herddb-dev at lists.herddb.org
> > http://lists.herddb.org/mailman/listinfo/herddb-dev
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://lists.herddb.org/pipermail/herddb-dev/attachments/20200117/b8d63a5d/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 2
> Date: Fri, 17 Jan 2020 09:46:40 +0000
> From: Alessandro Luccaroni - Diennea
>         <alessandro.luccaroni at diennea.com>
> To: Herddb developers <herddb-dev at lists.herddb.org>
> Subject: [Herddb-dev] R:  R: Herddb clients with go and python
> Message-ID: <298b5eb04aee4a978d2763122fcf26df at diennea.com>
> Content-Type: text/plain; charset="utf-8"
>
> I didn?t remember that we tried out Flatbuffers early on (with bad
> results)...if, indeed, the performance are not in the same ballpark (and,
> especially, if we hit the memory on the server side) maybe stick to the
> binary protocol is preferable.
>
> Alessandro
>
> Da: herddb-dev-bounces at lists.herddb.org <
> herddb-dev-bounces at lists.herddb.org> Per conto di Enrico Olivelli
> Inviato: venerd? 17 gennaio 2020 09:10
> A: Herddb developers <herddb-dev at lists.herddb.org>
> Oggetto: Re: [Herddb-dev] R: Herddb clients with go and python
>
> Alessandro,
>
> Il giorno ven 17 gen 2020 alle ore 09:04 Alessandro Luccaroni - Diennea <
> alessandro.luccaroni at diennea.com<mailto:alessandro.luccaroni at diennea.com>>
> ha scritto:
> We don?t have (as far as I know) many ?expert? in languages outside Java
> so I think going down the route of supporting protobuf/grpc is a good
> investment: if we get more traction it?s way easier for a newcomer to
> provide and support a new client this way.
>
> @Enrico: when you talk about ?performing the Java part? do you mean to
> switch also the Java client to grpc? If the performance are good/acceptable
> I think we should also evaluate to drop the custom binary protocol to
> streamline the code (this should be feasible right now since we have only
> one client implementation and we are still in the growing phase of the
> project)
>
> No, I mean to do the plumbing of the gRPC server into the server.
> Current "custom" protocol is super optimized in order to save as many in
> in memory copies as possible, both in the client and in the server,
> when the server receives a request it does need to create intermediate
> java objects from the parts of the request, it reads directly from the
> ByteBuf, the same is for the client.
> For a resultset the client receives a "big" ByteBuf and it reads the
> values directly from that buffer.
> Any other "higher level" abstract needs to create intermediate Java
> objects or add memory copies.
>
> So by now I don't like to drop current wire protocol.
> Maybe I am just biased by the experience with Flatbuffers we had in 0.7.0.
>
> Enrico
>
>
> Ale
>
> Da: herddb-dev-bounces at lists.herddb.org<mailto:
> herddb-dev-bounces at lists.herddb.org> <herddb-dev-bounces at lists.herddb.org
> <mailto:herddb-dev-bounces at lists.herddb.org>> Per conto di Enrico Olivelli
> Inviato: gioved? 16 gennaio 2020 22:02
> A: Herddb developers <herddb-dev at lists.herddb.org<mailto:
> herddb-dev at lists.herddb.org>>
> Oggetto: Re: [Herddb-dev] Herddb clients with go and python
>
> Amit
> Il gio 16 gen 2020, 19:30 Amit Chavan <achavan1 at gmail.com<mailto:
> achavan1 at gmail.com>> ha scritto:
> Hello,
>
> What does the team think about writing some clients for herddb in other
> languages?
> We can start with Go or python to begin with? Let me know your thoughts?
>
> It would be great.
>
> For network transport do we use anything specific with the current java
> implementation?
>
> We have a custom protocol, I think you already touched it.
> If we can support our native binary protocol it would be easier to get the
> same performances as for the java binding.
>
> We cam also add some alternative protocol using protobuf or even grpc.
> Maybe with grcp most of the wire protocol work is already done by build
> time plugins.
>
> If we want to go down the protobuf/grcp way I am happy to do the Java
> part. I know that Diego knows well grcp so maybe he can work on that topic
> as well (can't talk for me btw)
>
> I will start documenting the discussion and design approaches as we start
> having a discussion on this email thread.
>
>  Very good
>
>
> Enrico
>
> -Amit
> _______________________________________________
> herddb-dev mailing list
> herddb-dev at lists.herddb.org<mailto:herddb-dev at lists.herddb.org>
> http://lists.herddb.org/mailman/listinfo/herddb-dev
>
> ________________________________
>
> CONFIDENTIALITY & PRIVACY NOTICE
> This e-mail (including any attachments) is strictly confidential and may
> also contain privileged information. If you are not the intended recipient
> you are not authorised to read, print, save, process or disclose this
> message. If you have received this message by mistake, please inform the
> sender immediately and destroy this e-mail, its attachments and any copies.
> Any use, distribution, reproduction or disclosure by any person other than
> the intended recipient is strictly prohibited and the person responsible
> may incur in penalties.
> The use of this e-mail is only for professional purposes; there is no
> guarantee that the correspondence towards this e-mail will be read only by
> the recipient, because, under certain circumstances, there may be a need to
> access this email by third subjects belonging to the Company.
> _______________________________________________
> herddb-dev mailing list
> herddb-dev at lists.herddb.org<mailto:herddb-dev at lists.herddb.org>
> http://lists.herddb.org/mailman/listinfo/herddb-dev
>
> ________________________________
>
> CONFIDENTIALITY & PRIVACY NOTICE
> This e-mail (including any attachments) is strictly confidential and may
> also contain privileged information. If you are not the intended recipient
> you are not authorised to read, print, save, process or disclose this
> message. If you have received this message by mistake, please inform the
> sender immediately and destroy this e-mail, its attachments and any copies.
> Any use, distribution, reproduction or disclosure by any person other than
> the intended recipient is strictly prohibited and the person responsible
> may incur in penalties.
> The use of this e-mail is only for professional purposes; there is no
> guarantee that the correspondence towards this e-mail will be read only by
> the recipient, because, under certain circumstances, there may be a need to
> access this email by third subjects belonging to the Company.
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://lists.herddb.org/pipermail/herddb-dev/attachments/20200117/b055ae80/attachment.html
> >
>
> ------------------------------
>
> _______________________________________________
> herddb-dev mailing list
> herddb-dev at lists.herddb.org
> http://lists.herddb.org/mailman/listinfo/herddb-dev
>
>
> End of herddb-dev Digest, Vol 18, Issue 8
> *****************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.herddb.org/pipermail/herddb-dev/attachments/20200119/ddd38be6/attachment-0001.html>