diff --git a/minutes108_en.txt b/minutes108_en.txt new file mode 100644 index 0000000..47bbc39 --- /dev/null +++ b/minutes108_en.txt @@ -0,0 +1,23 @@ +Minuting – virtual coffee +29/09/2020 +40 min + +Purpose of the meeting: To discuss evaluation measure for shared task and improve communication procedure with annotator + +• [PERSON54] placed the coffee meeting link in.xz the google link to make suitable for all. +• Document of Minuting annotation guidelines contains detials of manully corrected transcripts and minutes. +• Discussion of various platforms suitable for coffee meetings. [PERSON7]'s zoom link for faculty will be most suitable as it is not time bounded. +• The data collection procdure involves first sennding the ASR transcripts to the annotators and correcting the data along with the audio or video file via FileSender(service provided by university) +• The stage one involves this transcript annotation by listening to the audio and correcting the transcript. +• The stage two involves writing the minutes and sending them back to [PERSON27]. +• The original minutes is shared with annotators only if they ask for it otherwise usually it is not shared. These are data copy from google document ad it contains the orginal agenda of the meeting, which is crated by [PERSON7] or [PERSON42] before thee meeting and partially extended during or after the meeting. +• The original minutes have drawback that they do not contain some information or contains some extra information which was not discussed in the meeting. They are reliable source of information of meeting. Though they can be reformed by changing the presentation; hence forming minutes for meeting. +• The orginal minutes may or maynot be present for certain meeting. As we [PERSON27] tries to get it from the organizers but sometimes meeting might be very short such as one line(LSD book where they discuss the book and not detailise it anymore). +• The stratergy of communication with annotator needs to be improved. As until now [PERSON27] sends transcript and original minute to annotators and they respond through email with manually corrected ASR and the minute. Further [PERSON27] puts them in server. The idea proposed by [PERSON7] is to self-employ the annotator to push and pull the data through GitHub. +• [PERSON55] does not agree with the idea of communicating with the annotator via Github but prefers the manual communication. +• The alignment of transcript and minutes would probably help in the evaluation of minutes. +• In shared task we need to specify whether we are manually or automatically evaluating the minutes. +• We might comeup with some measure using alignment otherwise we will use human evaluation such as readability and so on. +• [PERSON55] describes the data format needs meta data with brackets and in the summary keep bullets instead of dashes. +• [PERSON27] conveys that she has postpone the preparing of second basic sample until she discuss it with peter. If the annotation tool can work with the new samples or not. +• [PERSON55] suggest to annonymise the data before pushing it on Github. diff --git a/transcript108_en.txt b/transcript108_en.txt new file mode 100644 index 0000000..dc391b3 --- /dev/null +++ b/transcript108_en.txt @@ -0,0 +1,583 @@ +[PERSON27] Aha, so now I can do that. +[PERSON54] Yeah, right. +[PERSON27] Now I'm recording already. +(Fin, aha) I can see that. +And maybe I'll ask you before anybody else is (connecting), + how is it – how does it work with the link? +Do you create a next different link every time (with) – have a meeting? +[PERSON54] No, no, this is the same link you can find in the – +if you have accepted that meeting invite, it is the same link. +[PERSON27] , so (that's) – +[PERSON54] (Meetings) only. +[PERSON27] Yes. +So this Zoom link will also – will also function the next time we have the coffee meeting. +[PERSON54] Yea, yea, yea. +It is the – it is the (rolling) Zoom link, +we can use it like for – for all (of our meetings), so – +[PERSON27] Yes, okay, nice. +[PERSON54] For all our coffee meetings. +[PERSON27] Perfect, I just – +[PERSON54] So I'll also put – I'll also put the Zoom link as well as the (Google) link, so (??) is suitable for us. +We can +So in the same – same meeting (invite) you can find (??) (links). +[PERSON27] Okay, . +Thank you, thank you. +[PERSON54] Okay. +[PERSON27] And now let's wait before [PERSON53] or – a- and [PERSON55] are connecting. +[PERSON54] +(??) said that he had another meeting before a- +so from Doodle so – so – th- th- an hour before – before that. +Maybe he didn't finish yet, (I'm not sure). +[PERSON54] Okay. +So, [PERSON27], so (??) let discuss some things, okay? +[PERSON27] +[PERSON54] So, I – I wanted (to note) that – + so, (I didn't know) (??) – +it was like – +(I don't think there is) there is the – everything I (??) clean manually corrected transcript as well as (??). +Is that correct? + +[PERSON27] So our output is ma- manually corrected minutes and manually corrected transcript. +Is that what you ask(ed)? +[PERSON54] I – I – I want to know that how do you have (??)? +Like, how do the +[PERSON27] Okay. + Did you see the document Minuting annotation guidelines? +Probably – +[PERSON54] Yes. +[PERSON27] If I – +if we look at it together and I will just detailise some things that you want to know – +so , I will send you a link to minutes annotation, so that you can open it now. +[PERSON54] Okay. +[PERSON27] Do we have some chat here? +Yes. +[PERSON54] Maybe you can send it (to the hangnote). +[PERSON27] I have already sent it by chat, but I also can sent it by (hangnotes). +[PERSON54] Okay. +[PERSON27] (??) is right that Zoom is not working (for) (??). +How is it possible? +[PERSON54] Oh, yeah, how is it possible? +[PERSON27] So how is it possible that some link is working and another doesn't? +[PERSON54] I don't know. +(But) Zoom is like, you know, much bigger than Google meet. +[PERSON27] I also think – +[PERSON54] And we have problems. +[PERSON27] And especially I can hear much better with Zoom than with Google meet. +[PERSON54] Right, right, exactly. +[PERSON27] (One more) thing is – is that they want to make it – +so that – so we sh- we have to pay for it beginning (till) next month or so. +[PERSON54] Oh, (??)? +[PERSON27] Google meet. +It's not going to be (free any more). +[PERSON54] Oh, it is (already) free for one hour. +[PERSON27] Yes. +[PERSON54] But more then one hour (you have to pay). +[PERSON27] Oh, one hour, okay. +It's not (??) actually, okay. + So – +[PERSON54] But Google meet is also free for fourty minutes, but I want to (??) [PERSON27], how like [PERSON7] like (be have got like the paid for (??) Zoom). +Because (??) ([PERSON7] seems that) Zoom meeting link, it is (obvious) the – like +[PERSON27] We have (the paid) version of Zoom for our faculty. + But – +[PERSON54] Oh – +[PERSON27] There are some Zooms that we prepaid, but I'm not sure if it's a good option for lo- for our coffee m- coffee meetings. +Because that's kind of for everybody and (if) we have not cross with other – with other courses that are going on on – at our faculty. +[PERSON54] Okay, okay. + +[PERSON27] You really have to follow if nobody else is – s- needs this hour for their teaching, +so it's better to have something outside. +[PERSON54] So [PERSON27], could you – could you respond to (??), like if he can (reensolve) Zoom and (then join), because in Google meet it is really difficult to hear. + +[PERSON27] +[PERSON54] Okay. + Well, so, what we – we begin this data collection. + I will try to make a very short presentation how I do that. + So, we have automatic speech reconstruction transcripts. +So we have transcripts with really bad automatic quality, which I send to the annotator and ask him to correct the data. +And I also send him the audio file or video file via FileSender. + This is some service of our university that I can send big attachements. +I send a big attachement via some server and link to it, +and in the attachement of the e-mail i send him the pre-processed automatic speech reconstructed – speech reconstruction transcript. + So this is the stage one, transcript annotation, and he listens to the audio and corrects the transcript. +After that he writes from (stretch) the minutes and send them to me. +This is – +[PERSON54] (Hold on). +So when you send that – +so so what the annotator does is that he or she listens to the audio, and also goes to the automatic speech – speech transcript. +[PERSON27] Yes. +[PERSON54] And then? +He or she corrects that? +[PERSON27] Yes. +[PERSON54] Also based on (how they understand it)? +, am I right? +[PERSON27] He creates the minutes, yes. + Mostly I – +[PERSON54] Okay. +[PERSON27] (??) not to give them the original minutes. +So if they really ask for that, I can – I sometimes send them just for – +once they look like, but mostly – +[PERSON54] But +What is the original minutes? +[PERSON27] This is just what [PERSON7] created duri- +the agenda for the minutes. +[PERSON54] So this is the – like – (??) by the participant, great. +[PERSON27] Yes. +This is mostly – it – this is what was created by [PERSON7] within our [PROJECT1] minutes – [PROJECT1] meetings. +[PERSON27] So there are – +if you look at the data, there are a lot of docs there and data copy (pasted) from Google documents. + And the documents contain original agenda of the meeting, which was created by [PERSON7] or [PERSON42] before the meeting and partially s- (??) extended during the meetings, or sometimes after the meeting. +But this is kind of work version of what really was. +[PERSON54] Okay, so I would like to ask you now – +So, by this original meetings, what (??) [PERSON7] and [PERSON42] (has) created, so it is just for (reference of the annotators). +So (??) – +[PERSON27] This is just for what? +Sorry, once more please. +[PERSON54] This is just for the reference of the annotators. + Do you think it has any practical significance to our – our (??) or media (task)? +[PERSON27] I think it has, it may have. +It's not – +so this is also the output. +(??) doesn't have the (full) representation as needed yet, but it can be changed to this presentation and it also reflects the essence of the meeting. +so I cannot actually say that they are worse that the min- minutes created by annotators. +It is actually a questions, because there are some drawbacks of those originnal minutes, because they do not contain some information or they may contain something that was not discussed during the meeting. +But on the other hand they are rea , so they contain real names and they don't have errors in – in the content. +So actually they're not worse. +We can put them together with other – with transcripts and get more, just get more – +[PERSON54] So, not all the meetings has the original minutes, right? +[PERSON27] Yes, we try to get original minutes from the minu- meeting organisers, but sometimes it may be very short for example, for – +Some group of meetings, for example for LSD, the minutes are very very short and they contain just one line (LSD book). +And they discuss the book, not detailise it anymore. +[PERSON54] Okay. +Okay, so, so (ask) (??) (line) is good. +So, you send this (via file sender), and (they) (??) (you back) the manually corrected ASR? +And the minute for (??), right? +[PERSON27] Yes. +[PERSON54] Okay. +[PERSON27] And I put in – +I put it to the (server). +[PERSON54] You put it (??). +So now, (how did it work)? +So (did the teams) (??) directions? +Now (how did it work). +[PERSON27] Now it works the same, we din't – we didn't change it yet. +But after we change it – +So we didn't change anything yet, so actually , probably you have to change it. +[PERSON54] No. +I want to say that (support) after the team watch them, where from (today look the) (??). +(Today we look into the server, or should I have to be in touch with the annotator)? +[PERSON27] This is the question we discussed today in the morning. + So ([PERSON7]) proposes that annotators do that themselves. +We proposed that they will send the data to us. + So I still kind of – +so at the moment I do not have any idea hw to technically work with the variant that annotators send the data via (Github). +[PERSON7] – [PERSON7] is very much for that so that we really do that (Github) team and they can push and pull the data to the Github. + But at the moment I'm not ready to say how it should be organised, because I have to give them tasks s- +I have to say them 'Now you write this and that – this meeting' , I have to correct it, so I have to look if their transcript and the minutes are okay. +I have to provide – provide them with the audio and video file, +and this is an extra action that should be done apart from (Github), because it is a (hard) file. +[PERSON54] Okay. +[PERSON55] Sorry to interupt you Anja, I think that [PERSON7] already decided not to include – not to let them Yes. work with (GeteHub), because it deleted the – the team. +It deleted the team (??). +[PERSON27] Yes, but after (??) – +[PERSON55] But I think we – I think we agreed that they will keep communicating with you in the same way and that (??) will manage (log it up) – +[PERSON27] Well, well – +[PERSON55] (??) what you will – +[PERSON27] [PERSON55], not really, because afterwards, after that meeting, he said me many times h- that he still s- sees that it should be maybe changed. +So maybe not today but he still sees the option better that this (teams would be created) again, . +[PERSON55] Okay, so then we need another meeting I guess. +then we need another meeting with [PERSON7]. +[PERSON27] Not too long, but he- s kind of – +we spoke about that (Github) for a while after tha- afterwards. +[PERSON55] Okay. +[PERSON27] He – wha- he didn't try to persuade me to made these teams, but in his vision this teams still figurated somehow. +[PERSON55] Okay, okay. +[PERSON27] Probably (we should) – +[PERSON55] So, [PERSON27] – +[PERSON27] Not exclude this option for the moment, but – +[PERSON55] Okay, so, [PERSON27], I – yeah I – so – so this is like you to upload the minute (??) correct, so this is actually a (list) work for us. +So we sould be absolutely happy with that. +But the thing is, or the (only thing we worry) is, because since you (said) technical people, so if something goes wrong , otherwise we're absolutely fine with that. +[PERSON27] What is absolutely fine? +[PERSON55] Like if the annotators up- upload the minute themselves, so – +[PERSON27] Actually I'm not really fine with that at the moment, because for me – I should think it over and I didn't do it yet. +So I don't know how would I have the control o- over annotators. +The communication. +Because I have to send them what all the files manually, otherwise – +so I don't know how to do that. +I really have to upload the data and it takes time. +[PERSON55] I – I think the same [PERSON27], I absolutely agree with you. +I would not (ask) this persons to push in (Github) Yes. in the final repository to push there anything. +So I would also (gain) – say that I prefer this – the same strategy that you're using, (??) sending the data – +[PERSON27] So probably let's – yes. +Let's not do this team for the moment. +It could be changed, I think it could – +so, wh- I and [PERSON7] still thought about this option is not – not because of the present situation. +Now it's absolutely okay, that we just send them the data. +But he thought, overal the situation where they make alignments and it takes more time and somehow it can – it – it can – +it is connected with alignments. + +[PERSON55] Okay, okay. +[PERSON27] So let's not think about it at the moment if possible. +And if needed we would kind of build it on the- – on the existing system. +So at the moment I would prefer to have the contact with annotators, but actually I'm really Wonderful. good in that, so rather use (??). +[PERSON55] Okay. +[PERSON54] So, [PERSON27], so – so – let – [PERSON55] said in the morning – +I also like – +to some extent I thought and I – I think I also believe to some extent – +so – (??) we have an actually manual evaluation (??). +What is the – like, what is the vision (??). +[PERSON27] Aou want me to repeat why you need the alignment? +Why we need the al- +[PERSON54] No, no. +I – I need the al- +I – understood we need the alignment, but I was talk (??), so we must have a measure or a (??) by which we can use the alignment to produce something. +[PERSON27] +[PERSON54] Great, so, we do alignment, (??) fine, the manual (??) are done, but what is the final output? +It should be some (??) files and some nubers. +[PERSON27] Final oi- output of what. +[PERSON54] Final output of the alignment. +[PERSON27] (??), could I ask you, please, now, if you could write this very question very explicitly somewhere into – into the agenda. +[PERSON54] Okay. +[PERSON27] Because I think there should be a total understand– understanding between us what do you mean. +I'm not sure – +first I'm not sure that I understand what you ask and second I'm not sure – I'm not sure I can answer. +I'd rather think that answer should go from other source. +[PERSON54] Okay. +Okay, so I'm (??) typing in that – in that – +[PERSON27] Just form the question. +[PERSON54] – Google doc. +[PERSON27] +[PERSON54] So how are we going to use the alignment in evaluation? + +What is the measure or mid-check of this activity? +[PERSON27] Well, I – now I understand it. +I think it's a very good question and I don't think that we have answered it yet. +So we as- +[PERSON7] has kind of vague idea that it helps and so have I. + So this w- +so we have some experimen- experience with other tasks where such alignment helped somehow. +And we have the intuition based on some experiment- experiments with other data, that it would help. +Probably I'm a little bit sceptic now, but I – I really think that at the moment it is not solved. +So this is what – +if you would ask [PERSON7] this question he would probably say that (??) think about it and make o or invent the measure or metric for – for that. +So I've like – I – I would pick up from this (here), so the thing is that meeting or coming up with that measure want to be that much busy. +Like I'm just talking form (??) point of view. +So, probably we have to submit the proposal within the first week of November. +Okay, so, it – +I do see th- this coming within October, that I'll be (??) measure and that is accepted that, okay? +This is a manual measure for automatic (minority). +So the thing is that this is (??), right? +So, do you think that it could be (??) thing? +Because if we like – +I don't want a (??) that – +suppose we say about – +[PERSON27] Sorry, sorry, I lost the u- lost the idea. + Do you think that we should include what, evaluation measure? +[PERSON54] Do you think this alignment – +[PERSON27] Or alignment? +[PERSON54] First, alignment first, manual evaluation measure – +[PERSON27] Okay – +[PERSON54] Should be a part of the shared task? +[PERSON27] (??) I don't think about it. +I don't know, I have no idea. +So I really don't know. +I think this is the (??) you should discuss in more detail with [PERSON7]. +[PERSON54] +[PERSON27] Because – +[PERSON54] So I (??) this points. +[PERSON27] +[PERSON54] So this – +I would say – +[PERSON27] I have never had any experiments with shared task and measures and so on. +So I – what I am saying is kind of my intuition. + I Okay. cannot answer. +[PERSON54] Okay. +[PERSON27] Rather – +go- +either think with [PERSON53] about – about that or think with [PERSON53], [PERSON55] and [PERSON7], not me, I don't know. +[PERSON54] Yeah, okay, okay. +Okay, thank you. + Okay, so- suppose they does speak a scenario that we don't come up with a manual evaluation measure, before the shared task, so, what options we have. +So the options that we have is – we have the transcript, which is manually corrected, we have the minute, which is the (??), contains the (??) summary. +And then we (reuse) the data and we evaluate the submissions based on automatic measure. +[PERSON27] Do we – +where do we get the automatic measure? +[PERSON54] There are many automatic measures for (summarisation). +[PERSON27] But none of them works as ([PERSON53]) realised. +[PERSON54] How many do you have tried? +[PERSON27] How many what? +[PERSON54] How many – how many automatic measures have you tried? +[PERSON27] Ask [PERSON53]. +[PERSON54] Okay. +[PERSON27] Let – + so in summarising paper on that (??) maybe [PERSON55] also knows, none of them really works well. +[PERSON55] Well I wouldn't say none of them really works (well), I would say that they are kind of deceptive in some cases like they tch- they are very high – very high (??) – +[PERSON27] Yes . +[PERSON55] And then somebody (??) just (??) repeated this that – this that – this that, so you (??). +So they must be c- compo- – like associated with the human score. +Like (decives) them, we first add them, because we – they are very easy to compute. +But we add another human evaluation score, that was the point of th- that I wrote the protocol, like try to come up with some tips. +So, we need the na- next try human evaluation, (decides) them, we compute them, but we add the number of human evaluation, so that we can be sure that we have good quality. +That's the point. +[PERSON54] I agree with you. +But that human evaluation (??) should be like very subjective and (??), right? +[PERSON55] No, it's not very subjective, it's like – +we I – I think – +did you read the document that I wrote? +The protocol I wrote? +[PERSON54] Yes, yes. +[PERSON55] There is how most of the like evaluations are done in the community, in the literature. +Like have a summary or a text and we compare it with the reference and we have the annotators th- to compare it. +And they like try to assess different aspects like (??) reada- readibility, (the quality) etc., and they put scores of that. +And then we have this five stars for rating – (??) rating scale, from one to five stars. +And then we (abbregate), we wait this – let say this aspects, and we come up with the – with the final score. +This is the most common and easy way to – to do it (you know). +That's why I proposed that. +But then [PERSON7] has his own idea about this alignments and I don't really know, I know nothing about, so that's it. +[PERSON54] Yeah, yeah. +I – I – I got – I go- got the (multimission) of the (??) meeting script, but still that is like the (??) that [PERSON7] is targetting to achieve would you be more objective? +And the (liker giving) skills, okay, but still I would believe that it is subjective. +[PERSON55] Yes, I mean – +[PERSON54] (Right). + +[PERSON55] It's human based and every – every human evaluation is subjective, so – + +Everyone is doing it that way, so – +I mean most of the research evaluat- human evaluation in nature language (generation) is done that way, so- +[PERSON54] Okay, (wonderful). + + +Okay, so, like – +Okay, so the thing is that if we don't come up with the alignment based manual evaluation (??), we will do as the normally the other (energy) task (??) entries, right? +[PERSON55] I'm really sorry, I didn't follow you. +[PERSON54] Yeah, so So – if we – if we are like – if we are not able to come up with the alignment , if we are not able to come up with the alignment based manual evaluation measure, – +[PERSON55] Yes. +[PERSON54] We go with – we go with the (??), the +[PERSON55] Yes, sure, sure. +Yes, why not, yes. +That's – yes, okay, why not. +[PERSON27] Okay, so may I – may I summarise it, please? +So – +[PERSON55] Yeah. +[PERSON27] So really we want to use alignment annotation for summarisation, this is variant A. +But we have (little) time and very probably we don't have to prepare this alignments. +And in this case we follow variant B, and this is using human evaluation like readibility and so on, not using the alignment. +[PERSON54] Right. +[PERSON27] So – +[PERSON54] Yes. +Yes, so – so the thing is that my before November it would be difficult to come up with the alignment (this score), so for the (past) shared task that we have (way) to propose (??) speech, +I think we should go whatever is available and whatever is (better) to us. +And then maybe we would also like do some research on this and we come with some (beautiful) matrix, +and then maybe the second – you know the second event (??) we can come up with this new (??) – new techniques that we want to do. +Halo? +[PERSON55] So this – this – +you were talking to [PERSON27] or to me? +[PERSON54] I'm talking to you (??). +[PERSON27] +[PERSON55] So, I'm really confused. +[PERSON27] +Probably yes. +He just – you are just trying to summarise what we did and what we said, yes, yes. +[PERSON54] I'm just worried about like what the proposal should look like, because like for variant B there are still like some cloud on that. +We are not very clear. +So, I just want that the proposal should be (like very strong), and whatever we say shoould have a ground – grounded (this is to that). +(??) so, I now loose – +I know there are many autoatic measures for summarisation so if we like just put those in the proposal (no one we) question? +And we – if we like say that we are going to human evaluation using the like pst- like the reading skill or how they (??), (no one we yuestion)? +But if we are not (believed here) about the alignment , so that's it. +That's why my (??) (nice). +[PERSON27] I kind of don't know what to say. +So , I don't knw what you propose. +You propose directly to go plan – plan B, yes? +[PERSON54] No, B – we – like we have plan B as our backup. +If varian A doesn't become clear to us before the proposal submission deadline, we move in variant B. +If variant A is very clear, (its) concensus is accepted by all of us, we obviously go with variant B. +[PERSON27] Okay. +[PERSON54] Yeah. +Okay, okay. +Okay, so, yeah so I think – I think (??) no – (??) – +[PERSON27] you wanted to discuss something? +I have understood what is (??) that [PERSON7] was speaking about. +Yeah, it's how we should do the shared task and – +[PERSON27] Yeah, so – so, what I wanted to discuss is probably the format of the basic sample trial. +But this is – I think we should read (with) this – that format before [PERSON2] appears and discuss it with [PERSON2]. +[PERSON54] +[PERSON27] So this – +my – my point of how to create basic samples should be discussed with [PERSON2] before we go on. +So for now I don't prepare any basic samples more. +And then I – +wh- when [PERSON2] appears I will discussed – discuss with him, if his annotation tool can work with the (??) samples, +and then prepare the second version of basic samples. +Point? +And , as for the data format, this is also a point with [PERSON55] – +[PERSON55] As for the data format I'm already clear, like it's what we discussed like the three – three sections in each (file). +The first section is meta-data, you put on meta-data there, +and you're right in (??) brackets meta-data. +And then, the second (distrust) it, as I each (??), so the second and the third are crossed in the summary. +And in the summary you can keep the bullets, but only with (dashes). +[PERSON27] Yes. +[PERSON55] one dash, two dashes etc. +[PERSON27] Yes. +[PERSON55] So this is – I think this is the – +so it's almost like I said, but you can – +I said to remove the meta-data, +you can put them with – with a (label) in your (??) brackets – +[PERSON27] Yes, meta-data and dashes, dashes is important. +[PERSON55] So it's – it's almost the same, so the format is stable (??) – +[PERSON27] Yes, this is – +Well, I think I will have to discuss it with you before I begin once more about some details. +[PERSON55] Yes, sure, because – +yes, +[PERSON27] But, yes yes, but – +[PERSON55] But I think this is a – +because this way you put all the information, (you put practically put on) all information, because you have the meta-data (??) the summary. +There is nothing (??) – + +[PERSON27] Yes, yes. +So it's (good), isn't it? +(It was not) my point. +[PERSON55] (You deserve) the hierarchy of the bullets , but only with (??) recognisable, because (??) is not recognisable, so. +[PERSON27] Yes, yes, . +[PERSON55] I think for this we are okay, but we didn't have time in the morning to discuss about some other details, like anonymisation of – anonymi- anonymisation of the names of this annotators. +We need – we need to (encall) them, so you have to prepare a list with their real full names and their annot01, annot02, annot03 and that you have to put in the (folders). +So, you will have to rename (Zac). +You (will keep) a list, where (Zac) is like (Zac) what – I don't know is her name and then you put annot01, as a (sample) example. +And then you will use the zer- annot01 as the name to the folder structure that you – that we have, so you will have to rename the folder structure. +[PERSON27] [PERSON55], lwt's do it – +so, the question (??) do it when we do it who does it. + So, an I – +[PERSON55] What do you mean? +What – what – what, like renaming in the structure in the directory or the list, or what do you – +[PERSON27] So, now we are looki- +we are kind of changing the data storage from (Sorbow) to Github repository. +[PERSON55] No, we will keep both of them. +We'll keep the (??) – +[PERSON27] Yes, I know, but st- +yes. +But we are kind of centralizing it and we're making server as one of (mocking) versions of Github repository, so who is doig all of that? +[PERSON55] Once again, please, I couldn't hear you. +Where are restoring (??) – +[PERSON27] Who – who does all that Github things? +[PERSON55] For the Github it's only pushes, I (??) do it. +there is nothing to do in the Github for you. +You will just walk in the cluster. +What I was saying is that you or I or together we will have to rename the names in the cluster, all the names. +[PERSON27] Yes, yes – +[PERSON55] From (zac) – from (zac) you will put annot01, like anonymise the name. +[PERSON27] When? +Before (a create) in Github or it doesn't – or it doesn't – +[PERSON55] Before pushing, before pushing to Github. +[PERSON27] (Before) pushing? +This was my question. +[PERSON55] Yes, yes. +[PERSON27] So – +[PERSON55] Yes, sure, before pushing. +Because in Github it has to appear as anonymous, as annot- annot01. +[PERSON27] Yeah, so before pushing. +I have to change all the names, +so it means, that for each annotator I have to change the name form (zac)1, from (Odlova) to annot2 – +[PERSON55] Yes. +[PERSON27] Somebody – +so – and if I – +[PERSON55] Yes, so you create the (file) – you create the (file) the document that – internal – that you have access to, or even a notebook let say, a paper notebook, +and you have to keep the – the names of this collaborators or annotators, their full names – +[PERSON27] Yes, this is clear. +[PERSON55] And an anonymous name that is annot01, annot02, etc., yeah? +and you will (??) record of this list – +[PERSON27] Perfect, so this is okay, yes. +[PERSON55] (??) record of this list when you name the folders with their (collaborators). +[PERSON27] Yes, so this is – this is what – +[PERSON55] +[PERSON27] This is what I like. +If I – if I am allowed to do it so, I like it so. +Because I'm not able at the moment, I'm not sure if I can be – if I can make (??) but I – at the moment it is very complicated for me to anonymise all of the files to – so to name general names. +I have to keep the names – this final names of the files recognisable. +So it may – it should not be the name of the annotator, but it should be some ID that I can dete- that I can determine where to put it. +[PERSON55] Yes, I told you, annot01 is like – +[PERSON27] Yes, yes, in this case it is a (??), so nw my task is to rename the files. +[PERSON55] yes, but before that I also have – +I mean, I also propose some other things but you didn't have time to discuss them, so, I'm not sure – +[PERSON27] So, let's meet today, [PERSON55], let's meet today in person? +(??) to (??) (more) – +[PERSON55] [PERSON7] would be – better to have also [PERSON7], +I don't know, when he's available now, or tomorrow, or – +so maybe it's better to set up another meeting, like today – +like one of the day in the morning? +So, I think [PERSON7] – +[PERSON27] So, just (??) – +okay, okay. +So let's write – write [PERSON7] when he has time, so – +[PERSON55] Yes it would be very good. +[PERSON27] Or I can (??) Friday and (??) again, +I have (web seminars (??) Thursday – +[PERSON55] Probably, tomorrow morning or evenyou want today afternoon or some time? +It would be very good to have also [PERSON7] (to seome things). +[PERSON27] Yes. +[PERSON55] Because I'm really worried about this team that he's proposing, so, +I (??) disagree with [PERSON7] about several things, so we nedd to discuss (??). +[PERSON27] Yes, (??) be everybody (??) in person, yes, perf- + I agree. +[PERSON27] So – +[PERSON55] So – +[PERSON27] I'll write him and you now and – +[PERSON55] Yes, try to – we try to catch him. +[PERSON27] (??) meeting tomorrow morning. +[PERSON55] +we try to catch him as soon as possible. +[PERSON27] Yes, not today, I am already at home – +What? +[PERSON55] Okay. + the meeting. +I will – +[PERSON55] Yes, sure. +[PERSON27] Okay. +[PERSON54] Yes, (??) something like the meeting of today, you were also present (??) – +[PERSON55] Yeah. +[PERSON54] Repeat the one today. +[PERSON27] Okay. +[PERSON54] Okay, okay. +Okay, so now we're just – +like everything is like almost sorted. +the thing is that we have to like wait for [PERSON2] (??) to actually (??) that – (??) this annotation tool composed (for new) (??) (idea). +[PERSON27] Yeah, [PERSON53] is writing. +I was waiting and tried install Zoom again and with different e-mail adress. +Now it asking me meeting password. +Could someone please provide (??) to my lock? +(Do you) let me know if something concers me? +He needs some password to (??), what does it mean? +Which password? +Meeting password, what is that? +[PERSON54] I think the meeting pass- +yeah, the meeting password is minuting, which is already there in the meeting (??) +[PERSON27] Could you – could it – could you please send it to [PERSON53]? +[PERSON54] Yeah, yeah, (??). +[PERSON27] But anyway, I would rather probably finish it for today. +[PERSON54] Yeah, okay. +I (too) think so. +[PERSON27] +[PERSON54] ([PERSON55]), anything else you want to discuss? +[PERSON27] No. +[PERSON55] No, I mean I definitely – (with) definitely (??) with [PERSON7] to – +because today we didn't finish, like we left some issues without dicsussing them, so – +We cannot really discuss anything without [PERSON7], because – +[PERSON54] Exactly, yeah. +There are like decission points, decisions points has to take, so yeah. +[PERSON55] So, there is nothing to discuss today anymore. +Okay – +[PERSON27] So, have a nice evening, everybody. +[PERSON55] Okay, goodbye. +[PERSON54] (Yeah). +[PERSON27] Bye bye. +[PERSON54] Goodbye, and stay (??). +Bye bye. +[PERSON27] You too. + + + + + + +I 'm not, you know, wanting is like, yes? +Yes, in this case, it is he. + +So now my task is to return...