[Herddb-dev] Thank you for bookeeper info
Amit Chavan
achavan1 at gmail.com
Mon Mar 25 06:07:24 CET 2019
Hello Enrico,
Thanks for sending information about bookkeeper to me. I went through that
tutorial and played around with the bookkeeper a little bit. I have few
things I want to confirm -
1. Herddb uses bookkeeper[BK] to ensure that writes/updates/deletes made to
the database are replicated and we don't lose them
2. If we use BK as a WAL then where does the actual data in the table
spaces reside. I am guessing the WAL is used by Herddb to keep track of
individual inserts, edits, deletes which are first written to BK and
applied to in memory copy. Does the data only reside in-memory or it ever
makes to the disk outside of BK?
4. What happens to the log entries in the ledgers held within bookkeeper.
Are they only kept for recovery if the in-memory herddb application crashes?
5. What is the policy for retention of ledgers ? Do we keep one ledger per
table space?
I will start work on the jdbc UT tonight.
Thanks,
Amit
On Fri, Mar 22, 2019 at 4:00 AM <herddb-dev-request at lists.herddb.org> wrote:
> Send herddb-dev mailing list submissions to
> herddb-dev at lists.herddb.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://lists.herddb.org/mailman/listinfo/herddb-dev
> or, via email, send a message with subject or body 'help' to
> herddb-dev-request at lists.herddb.org
>
> You can reach the person managing the list at
> herddb-dev-owner at lists.herddb.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of herddb-dev digest..."
>
>
> Today's Topics:
>
> 1. Re: herddb-dev Digest, Vol 9, Issue 1 (Amit Chavan)
> 2. Re: herddb-dev Digest, Vol 9, Issue 1 (Enrico Olivelli)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 21 Mar 2019 16:35:30 -0700
> From: Amit Chavan <achavan1 at gmail.com>
> To: herddb-dev at lists.herddb.org
> Subject: Re: [Herddb-dev] herddb-dev Digest, Vol 9, Issue 1
> Message-ID:
> <CADsOBjOc4COf3=
> QVg46FLOc38NOjsDtJw_9C2kL7D383EFJW5w at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi Enrico,
>
> I will look at issues tagged to me. Any good information on how we use
> bookkeeper to store data will be useful too. Do we have calcite adapter to
> work with bookkeeper?
>
> Thanks,
> Amit
>
> On Thu, Mar 21, 2019 at 4:00 AM <herddb-dev-request at lists.herddb.org>
> wrote:
>
> > Send herddb-dev mailing list submissions to
> > herddb-dev at lists.herddb.org
> >
> > To subscribe or unsubscribe via the World Wide Web, visit
> > http://lists.herddb.org/mailman/listinfo/herddb-dev
> > or, via email, send a message with subject or body 'help' to
> > herddb-dev-request at lists.herddb.org
> >
> > You can reach the person managing the list at
> > herddb-dev-owner at lists.herddb.org
> >
> > When replying, please edit your Subject line so it is more specific
> > than "Re: Contents of herddb-dev digest..."
> >
> >
> > Today's Topics:
> >
> > 1. Bad error from CI (Enrico Olivelli - Diennea)
> > 2. Request - More tasks (Amit Chavan)
> > 3. Re: Request - More tasks (Enrico Olivelli)
> >
> >
> > ----------------------------------------------------------------------
> >
> > Message: 1
> > Date: Wed, 20 Mar 2019 14:45:29 +0000
> > From: Enrico Olivelli - Diennea <enrico.olivelli at diennea.com>
> > To: "herddb-dev at lists.herddb.org" <herddb-dev at lists.herddb.org>
> > Subject: [Herddb-dev] Bad error from CI
> > Message-ID:
> > <30102e56596cf23f1041b95317a54a3d7bed3461.camel at diennea.com>
> > Content-Type: text/plain; charset="utf-8"
> >
> > Hi guys, we are very close to 0.9.0 release,
> > but today I found this bad error on Jenkins...
> >
> > https://github.com/diennea/herddb/issues/361
> >
> > We have to fix it before a release
> >
> >
> > --
> >
> > Enrico Olivelli
> > Software Development Manager @ Diennea - MagNews
> > Tel.: (+39) 0546 066100 - Int. 125
> > Viale G.Marconi 30/14 - 48018 Faenza (RA)
> >
> > ________________________________
> >
> > CONFIDENTIALITY & PRIVACY NOTICE
> > This e-mail (including any attachments) is strictly confidential and may
> > also contain privileged information. If you are not the intended
> recipient
> > you are not authorised to read, print, save, process or disclose this
> > message. If you have received this message by mistake, please inform the
> > sender immediately and destroy this e-mail, its attachments and any
> copies.
> > Any use, distribution, reproduction or disclosure by any person other
> than
> > the intended recipient is strictly prohibited and the person responsible
> > may incur in penalties.
> > The use of this e-mail is only for professional purposes; there is no
> > guarantee that the correspondence towards this e-mail will be read only
> by
> > the recipient, because, under certain circumstances, there may be a need
> to
> > access this email by third subjects belonging to the Company.
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL: <
> >
> http://lists.herddb.org/pipermail/herddb-dev/attachments/20190320/945e8aba/attachment-0001.html
> > >
> >
> > ------------------------------
> >
> > Message: 2
> > Date: Thu, 21 Mar 2019 00:54:06 -0700
> > From: Amit Chavan <achavan1 at gmail.com>
> > To: herddb-dev at lists.herddb.org
> > Subject: [Herddb-dev] Request - More tasks
> > Message-ID:
> > <CADsOBjPo8G0N_=
> > rzdjVfatokERNE5krohz09Gv8hjknOsv2pQQ at mail.gmail.com>
> > Content-Type: text/plain; charset="utf-8"
> >
> > Hello,
> >
> > I finished https://github.com/diennea/herddb/pull/359 and looking to
> take
> > up more tasks.
> >
> > Thanks,
> > Amit
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL: <
> >
> http://lists.herddb.org/pipermail/herddb-dev/attachments/20190321/3fb6ba36/attachment-0001.html
> > >
> >
> > ------------------------------
> >
> > Message: 3
> > Date: Thu, 21 Mar 2019 08:58:59 +0100
> > From: Enrico Olivelli <eolivelli at gmail.com>
> > To: Herddb developers <herddb-dev at lists.herddb.org>
> > Subject: Re: [Herddb-dev] Request - More tasks
> > Message-ID:
> > <
> > CACcefgee+MLGsXGBxDiSZjX4guvMJx81jprWY-m44MFSE+Nn8Q at mail.gmail.com>
> > Content-Type: text/plain; charset="UTF-8"
> >
> > Amit,
> > thank you for your contributions!
> > I think we have to push down some unit test in the jdbc module
> > regarding the "NOT NULL" feature.
> >
> > So:
> > - positive/negative tests around "create table"
> > - JDBC DatabaseMetadata#getColumns
> > - positive/negative tests around INSERTS on not null columns
> > - positive/negative tests around UPDATE on not null columns
> > - simple scan over not null columns
> > - order by order not null columns
> > - where clauses over not null columns
> > - basic join over not null columns
> >
> > We have a full set of new data types, we have to ensure that these new
> > datatypes work well on every common usage pattern
> >
> > I have tagged you on other issues,
> > If you have preferences over some topic please tell,
> > I think you are ready to work on more complex tasks
> >
> > Cheers
> > Enrico
> >
> >
> > Il giorno gio 21 mar 2019 alle ore 08:55 Amit Chavan
> > <achavan1 at gmail.com> ha scritto:
> > >
> > > Hello,
> > >
> > > I finished https://github.com/diennea/herddb/pull/359 and looking to
> > take up more tasks.
> > >
> > > Thanks,
> > > Amit
> > > _______________________________________________
> > > herddb-dev mailing list
> > > herddb-dev at lists.herddb.org
> > > http://lists.herddb.org/mailman/listinfo/herddb-dev
> >
> >
> > ------------------------------
> >
> > _______________________________________________
> > herddb-dev mailing list
> > herddb-dev at lists.herddb.org
> > http://lists.herddb.org/mailman/listinfo/herddb-dev
> >
> >
> > End of herddb-dev Digest, Vol 9, Issue 1
> > ****************************************
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://lists.herddb.org/pipermail/herddb-dev/attachments/20190321/da9229eb/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 2
> Date: Fri, 22 Mar 2019 08:22:46 +0100
> From: Enrico Olivelli <eolivelli at gmail.com>
> To: Herddb developers <herddb-dev at lists.herddb.org>
> Subject: Re: [Herddb-dev] herddb-dev Digest, Vol 9, Issue 1
> Message-ID:
> <CACcefgf7dBZ26yDwM4BQWxMC+1SekhSVsgskXu_Vmsu5=
> 8sASA at mail.gmail.com>
> Content-Type: text/plain; charset="UTF-8"
>
> Amit,
> Here is where we are dealing with BookKeeper
>
>
> https://github.com/diennea/herddb/blob/master/herddb-core/src/main/java/herddb/cluster/BookkeeperCommitLog.java
>
> We are using BookKeeper as Write-Ahead-Log for the database, every
> write is written to the WAL and then the change is applied in memory.
>
> BookKeeper replicates data and it makes really simple to build replication.
>
> Foreach tablespace you have a "leader" which receives all of the
> queries, and a number of "followers" (replicas) which are "tailing"
> the WAL and applying the same set of changes in the same order to the
> local copy of data.
>
> It is possible that the leader dies or is it partitioned from the
> cluster, in this case any follower can be elected as leader.
> BookKeeper guarantees (thru the 'fencing' feature) that the system
> will always be consistent:
> - only one leader can make progress
> - every entry read from the log will be always available for reads
>
> you can check out BK docs:
> https://bookkeeper.apache.org/docs/4.9.0/getting-started/concepts/
>
> this is a Tutorial about BookKeeper which demonstrates its basic usage as
> WAL:
> https://github.com/ivankelly/bookkeeper-tutorial
>
> hope that helps
> Enrico
>
> Il giorno ven 22 mar 2019 alle ore 00:36 Amit Chavan
> <achavan1 at gmail.com> ha scritto:
> >
> > Hi Enrico,
> >
> > I will look at issues tagged to me. Any good information on how we use
> bookkeeper to store data will be useful too. Do we have calcite adapter to
> work with bookkeeper?
> >
> > Thanks,
> > Amit
> >
> > On Thu, Mar 21, 2019 at 4:00 AM <herddb-dev-request at lists.herddb.org>
> wrote:
> >>
> >> Send herddb-dev mailing list submissions to
> >> herddb-dev at lists.herddb.org
> >>
> >> To subscribe or unsubscribe via the World Wide Web, visit
> >> http://lists.herddb.org/mailman/listinfo/herddb-dev
> >> or, via email, send a message with subject or body 'help' to
> >> herddb-dev-request at lists.herddb.org
> >>
> >> You can reach the person managing the list at
> >> herddb-dev-owner at lists.herddb.org
> >>
> >> When replying, please edit your Subject line so it is more specific
> >> than "Re: Contents of herddb-dev digest..."
> >>
> >>
> >> Today's Topics:
> >>
> >> 1. Bad error from CI (Enrico Olivelli - Diennea)
> >> 2. Request - More tasks (Amit Chavan)
> >> 3. Re: Request - More tasks (Enrico Olivelli)
> >>
> >>
> >> ----------------------------------------------------------------------
> >>
> >> Message: 1
> >> Date: Wed, 20 Mar 2019 14:45:29 +0000
> >> From: Enrico Olivelli - Diennea <enrico.olivelli at diennea.com>
> >> To: "herddb-dev at lists.herddb.org" <herddb-dev at lists.herddb.org>
> >> Subject: [Herddb-dev] Bad error from CI
> >> Message-ID:
> >> <30102e56596cf23f1041b95317a54a3d7bed3461.camel at diennea.com>
> >> Content-Type: text/plain; charset="utf-8"
> >>
> >> Hi guys, we are very close to 0.9.0 release,
> >> but today I found this bad error on Jenkins...
> >>
> >> https://github.com/diennea/herddb/issues/361
> >>
> >> We have to fix it before a release
> >>
> >>
> >> --
> >>
> >> Enrico Olivelli
> >> Software Development Manager @ Diennea - MagNews
> >> Tel.: (+39) 0546 066100 - Int. 125
> >> Viale G.Marconi 30/14 - 48018 Faenza (RA)
> >>
> >> ________________________________
> >>
> >> CONFIDENTIALITY & PRIVACY NOTICE
> >> This e-mail (including any attachments) is strictly confidential and
> may also contain privileged information. If you are not the intended
> recipient you are not authorised to read, print, save, process or disclose
> this message. If you have received this message by mistake, please inform
> the sender immediately and destroy this e-mail, its attachments and any
> copies. Any use, distribution, reproduction or disclosure by any person
> other than the intended recipient is strictly prohibited and the person
> responsible may incur in penalties.
> >> The use of this e-mail is only for professional purposes; there is no
> guarantee that the correspondence towards this e-mail will be read only by
> the recipient, because, under certain circumstances, there may be a need to
> access this email by third subjects belonging to the Company.
> >> -------------- next part --------------
> >> An HTML attachment was scrubbed...
> >> URL: <
> http://lists.herddb.org/pipermail/herddb-dev/attachments/20190320/945e8aba/attachment-0001.html
> >
> >>
> >> ------------------------------
> >>
> >> Message: 2
> >> Date: Thu, 21 Mar 2019 00:54:06 -0700
> >> From: Amit Chavan <achavan1 at gmail.com>
> >> To: herddb-dev at lists.herddb.org
> >> Subject: [Herddb-dev] Request - More tasks
> >> Message-ID:
> >> <CADsOBjPo8G0N_=
> rzdjVfatokERNE5krohz09Gv8hjknOsv2pQQ at mail.gmail.com>
> >> Content-Type: text/plain; charset="utf-8"
> >>
> >> Hello,
> >>
> >> I finished https://github.com/diennea/herddb/pull/359 and looking to
> take
> >> up more tasks.
> >>
> >> Thanks,
> >> Amit
> >> -------------- next part --------------
> >> An HTML attachment was scrubbed...
> >> URL: <
> http://lists.herddb.org/pipermail/herddb-dev/attachments/20190321/3fb6ba36/attachment-0001.html
> >
> >>
> >> ------------------------------
> >>
> >> Message: 3
> >> Date: Thu, 21 Mar 2019 08:58:59 +0100
> >> From: Enrico Olivelli <eolivelli at gmail.com>
> >> To: Herddb developers <herddb-dev at lists.herddb.org>
> >> Subject: Re: [Herddb-dev] Request - More tasks
> >> Message-ID:
> >> <
> CACcefgee+MLGsXGBxDiSZjX4guvMJx81jprWY-m44MFSE+Nn8Q at mail.gmail.com>
> >> Content-Type: text/plain; charset="UTF-8"
> >>
> >> Amit,
> >> thank you for your contributions!
> >> I think we have to push down some unit test in the jdbc module
> >> regarding the "NOT NULL" feature.
> >>
> >> So:
> >> - positive/negative tests around "create table"
> >> - JDBC DatabaseMetadata#getColumns
> >> - positive/negative tests around INSERTS on not null columns
> >> - positive/negative tests around UPDATE on not null columns
> >> - simple scan over not null columns
> >> - order by order not null columns
> >> - where clauses over not null columns
> >> - basic join over not null columns
> >>
> >> We have a full set of new data types, we have to ensure that these new
> >> datatypes work well on every common usage pattern
> >>
> >> I have tagged you on other issues,
> >> If you have preferences over some topic please tell,
> >> I think you are ready to work on more complex tasks
> >>
> >> Cheers
> >> Enrico
> >>
> >>
> >> Il giorno gio 21 mar 2019 alle ore 08:55 Amit Chavan
> >> <achavan1 at gmail.com> ha scritto:
> >> >
> >> > Hello,
> >> >
> >> > I finished https://github.com/diennea/herddb/pull/359 and looking to
> take up more tasks.
> >> >
> >> > Thanks,
> >> > Amit
> >> > _______________________________________________
> >> > herddb-dev mailing list
> >> > herddb-dev at lists.herddb.org
> >> > http://lists.herddb.org/mailman/listinfo/herddb-dev
> >>
> >>
> >> ------------------------------
> >>
> >> _______________________________________________
> >> herddb-dev mailing list
> >> herddb-dev at lists.herddb.org
> >> http://lists.herddb.org/mailman/listinfo/herddb-dev
> >>
> >>
> >> End of herddb-dev Digest, Vol 9, Issue 1
> >> ****************************************
> >
> > _______________________________________________
> > herddb-dev mailing list
> > herddb-dev at lists.herddb.org
> > http://lists.herddb.org/mailman/listinfo/herddb-dev
>
>
> ------------------------------
>
> _______________________________________________
> herddb-dev mailing list
> herddb-dev at lists.herddb.org
> http://lists.herddb.org/mailman/listinfo/herddb-dev
>
>
> End of herddb-dev Digest, Vol 9, Issue 2
> ****************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.herddb.org/pipermail/herddb-dev/attachments/20190324/1a3dc7e3/attachment-0001.html>
More information about the herddb-dev
mailing list