How to Spot a Deepfake (Explained by an AI Vision Expert)

For content authenticity which is called C2PA. Google, Adobe, a lot of these different big players have now adopted these standards. So we are really good at physical vision. I'm not as sure as we are that good at digital vision. Today's AI can do a lot. I think the damage can be much worse, right? We all saw that from the Grock uh AI example uh where you know early in January uh it was being used to undress you know people online right

AI is it hype or help? Good day and welcome to the show. I'm your host Pete Cooper and our topic is AI hype or help. Today's guest is Apana Barati. She is a PhD in AI, machine learning and vision systems.

A very important topic in today's world as we're seeing so many quick advancements in this field. It also happens to be a field that I share a common interest in and a partner and I actually work together on a project. So this is going to be a deep dive into AI vision systems and machine learning. So let's get into it. Welcome Apana. Welcome. Thank you for joining us today.

Thank you so much. Thank you for the invitation.

Yeah, I've been looking forward to this one a partner because we work together on uh some AI vision technology and I wanted to share that with our guests and get them to know a little bit about what we're doing u what AI vision is and what it's about and importantly what what you bring to the table which is a lot. Um so a partner why don't you start with a little bit of your background and what you do in AI vision.

Yeah so um I am an assistant professor at Lehi right now and uh my research interests are in the areas of computer vision, machine learning and AI. Um and uh I develop technology that uh is generally related to learning from visual data. Right?

So in in wherever we have problems that involves uh images and videos and uh uh and you know and maybe some metadata associated with it uh we try to build those you know u models uh using that type of modality

uh to solve various different problems right so it could be problems ranging from visual recognition to scene understanding to fighting misinformation um etc.

So uh I think there is uh definitely uh a quite a lot of interest that I have in also security related applications where we want to you know think about how do we protect really high value media assets right so whether it is with watermarks or with other kinds of technology how do we ensure some sort of traceability

um when uh when original data and real human generated data is being used in the training of these AI models and things like that.

Very hot topic Aana isn't it? I mean uh I think everyone who listens to this podcast would be aware that AI can generate some pretty convincing image data and it's only going to get more and more convincing.

Yes.

Possibility for mis you know misinformation, right?

Yes. I think it is already there. Um, so I think a lot of it is like so of obviously that depends on the prompt and what do you intend to generate but these models have gotten really good in their quality of generation and um you know and uh u they seem to be getting even better. So there is a lot of uh um you know um motivation to to detect some of these generated images and videos

and also analyze things further about you know how much of original content do they copy and things like that. So yeah.

So if I go back even just 12 months

I was looking at this classic video of um of the guy eating spaghetti you know and it just doesn't look right. Um, Will Smith eating spaghetti. If you look it up, it's him eating spaghetti, but it's all autogenerated, but it's a mess and things are just going all over the place. It's totally weird.

Now, the latest ones are almost to the point where you can't tell that it's AI generated. It really looks like Will Smith eating spaghetti.

Yes. And I think a lot of it um has to also be about context, right?

So how realistic is it uh for you know Will Smith to be posting a video of him eating spaghetti right so so there's a lot of um that is not just captured in the image or the video but you know for a human viewer who is looking at it uh they might have a lot more context from just you know knowing things historically or anecdotally and things like that. So

um so those are the ones that are really hard to capture in this process of fighting misinformation

right.

So we can look at pixel level artifacts and we can look at you know generation information of these media objects and uh but the the the interaction with the h human viewer is where there are a lot of interesting research questions.

Pixel what did you call it pixel level um that again yeah

pixel level artifacts artifacts

that sounds very interesting tell me a bit more about that

yeah so um so as we know that images are represented as a you know matrix of numbers in a computer's memory and uh um so these numbers have statistical patterns right uh some of it uh is obviously from you know when you take the picture from the camera and then you have uh these kind of photons then converted into electrons

you record that charge in the form of a number right which is then stored on the your SD card or whatever memory device you have.

Um so in that process of image acquisition of when we are taking a picture there are different uh parts of that cycle right that uh have some level of quantization or there is some sort of sensor noise because not all manufactured sensors will behave you know in a specific way or behave the same way right so each one of them behaves distinctly

so there are statistical methods to extract such patterns. So we know that if there is an image that has been acquired by the camera, it is going to have those underlying noise patterns. Right.

Fascinating.

Whereas Yeah.

Whereas if there is an uh you know image or a video that has been generated, while it might look like the scene kind of makes sense, it may not have some of those underlying details.

Amazing. So I worked in cameras.

I think we've discussed this way back in the '9s when people used to say why would you want to have a camera in a cell phone um you know because they felt that it wasn't necessary but now we have multiple cameras I think my four maybe five cameras my phone um cameras everywhere um

so I got quite deep into the technology around you know trying to get the most out of these very early sensors um a lot of it's been overcome because of just the uh the noise is is is buried in just so many pixels you don't see. It's not so big. But also they they've con constantly improved the um the pixel capabilities.

I mean at the end of the day the photon of light goes in and and hits the the pixel and um

creates a bit of charge which is then turned into a voltage. Um and that uh isn't perfect as you said and it create there's a little bit of noise already in the in every system and particularly in in this and it has certain characteristics which um prove in a way that it came from a real sensor and even a particular manufacturer's sensor

sensors yes uh I think uh uh that is very well captured uh about the process of you know image acquisition through a camera.

Um whereas now I think there are there is to some level you can replicate some of these noise patterns when we try to use modern AI generators

to generate images and videos. But it is not always the case, right? Because you are trying to now replicate not just the scene composition through just you know a sentence or a prompt. Um now you have to specify you know quite a few number of constraints to make sure that you get a realistic looking image uh that uh is you know maybe physically consistent with the knowledge of the world that the viewer has uh in addition to having these really you know small uh and pixel level statistical fingerprints.

So this is making the task of the generator very difficult.

So you know the the chances of them you know failing at one of these constraints is now higher. So I think even though people say oh we can easily you know model the generator to replicate these things then we know that you know this is uh us trying to make the task of the generator harder. There's a classic arms race between those that want to I guess a full

generate the images to make it look realistic the the real world those that want to capture the real world. Um are the sensor manufacturers you know being proactive with that trying to to help us ensure that we're getting real images with uh those techniques?

Yes.

So I think the industry is very you know aware of these problems and uh I think it uh it was Qualcomm that uh that started kind of in some way watermarking a bunch of their u images right so in whatever phones that used their specific processor you're going to have some level of traceability right that this came from this specific device so they do it like you know from the get- go.

So, so that is there there is there are a bunch of uh industry members that have come together and they have found they have formed this kind of coalition uh for content authenticity which is called C2PA

right so uh I think Google Adobe a lot of these different big players have now adopted um this these standards and they want like you know the media that that is theirs right or has been creat created through their uh software or tool has this level of protection in some way.

So when you say a watermark um I think most people would see that as a something that's sort of visible but not highly visible.

So if you take a sheet of paper, you see the printer text and you can see behind a background which is the watermark. We don't want that in an image. You don't want it visible. So it's an invisible watermark.

Yes. Yes. So I think mo most of these are invisible watermarks. So think of it as kind of digital signatures that are embedded in the image but uh uh you know are not perceptible to the human eye.

Well that's really fascinating and I think it gives me some comfort that there is uh some way to try and uh thwart those that are trying to fool us. Right. I don't think anybody wants to be fooled.

I guess I was looking at something literally only yesterday about a cr you know these big tower cranes on construction sites that were collapsing and some of them looked not real to me and I was really trying to figure out why it didn't look real. My eye was or my sense was there, but it took me quite a while to realize specifically what it was that was jarring me. And I still at the end of it, I still wasn't sure whether it was autogenerated or it was just a bit of motion blur because of course things that move quickly in an image create a kind of blurring streaking effect.

Um, and um, it's I'm not sure whether that's because of a camera artifact or it's because of an autogenerated artifact. Yeah, I think this is a very difficult problem. So, uh I think vision for humans is is is very good, right? It's very advanced, right? You you look at something, you know what to do with it. When you are e, you know, we are very um capable of navigating our own surroundings and uh you know recognizing people and objects and things like that. But our visual system was not trained to discern you know what has been generated and what has been digitally manipulated and so on.

So we are really good at physical vision.

I'm not as sure as we are that good at digital vision but nowadays there is a lot of digital vision in play because we are all on screens all the time right and a lot of our uh the information that we get about the world

is coming through this medium.

So unfortunately you know that is not something that we are great at. So humans may not be able to tell you know more than a a random chance uh whether something is real or something was you know manipulated.

Yeah.

Um yeah. So

I think also there's the sensor guys you mentioned some of the bigger players like Adobe but what about those that present the data to us?

You know the YouTubes and the Vimeos and the other platforms that present that are they doing anything to try and help us discern reality from from the fake.

Yeah. So I think uh u so YouTube is uh is you know u so they have to consider video watermarking or like you know other kinds of technologies that uh that are uh applicable to long- form content. So that is another kind of you know problem because you just now have this huge amount of data right that you want to now embed reliable signatures into and things like that. So I'm sure it is on the way.

Um but uh there are uh various different platforms that do um you know add their own signature if you upload content to that platform and uh um and we have seen this through various apps right. So there was this kind of Sora 2 app from OpenAI and they would always have this kind of uh slightly visible in their case watermark because of this kind of misinforming implications. Right? So,

so Sora 2 for everyone is in generating autogenerating images or videos or

videos. Yeah, videos.

Okay. So, they they uh open AI deliberately declaring that something has been autogenerated. So, you can use it for fun over here, but don't use it to try and to fool us over here. Right.

Yeah.

But but the same technology can be right. I mean this one that they released as an app was supposed to just be used for you know um funs and giggles and so on but the same technology can be used to actually create misleading content right and uh

so so that is the the scary part is that you know if you don't have this extra layer um and that is where I think this is not a problem of just technology right so this is a problem of coming up with policies. This is a problem of regulation. Uh because you know we've always had technologies that can be used for nefarious activities. Um this is definitely more powerful than a lot of the other technology that we have seen so far.

Um you know so today's AI can do a lot. Um I think the damage can be much worse and it can be very quick. Right? We all saw that from the Grock uh AI example uh where you know early in January uh it was being used to undress you know people online right so if you have a photograph and people were using the technology to create you know nude versions of those people

so I think uh um you know before the uh the administrators intervened there was already quite a few I would say in millions a number of images that had already been you know uh attacked in some way. Attack is what I say but like you know edited in some way. Yeah.

Yeah. Yeah.

I mean if you're a victim of that it would be quite distressing to have you know to have that. Um and I guess you know people who are in this spotlight famous people or people that are well known I guess in a way are particularly vulnerable um to that type of attack. Um I mean it's with as with all technology there is always the good and the bad isn't there? Um

and as you say you know it's this is particularly nefarious um because it can do a lot of damage I guess quite easily. You know, it's not not that hard to do that type of thing.

Yes. Yes. We no longer need Photoshop experts or like you know, people who know how these media technologies work. Right.

Now, everybody can just write a sentence like, you know, a prompt, an instruction

uh and then try to, you know, create something that looks very realistic. M

so are you seeing any any new policies coming out that I mean we're here in the US but in Europe as well have you seen anything coming out which is trying to address that that problem.

So uh I think there is the uh some uh sort of dialogue uh regarding copyrights.

So the number of copyright cases have definitely increased you know since uh and we have like we have heard about some of the high-profile ones um such as between stability AI and uh you know um um midjourney and uh there is uh um there's like different kinds of lawsuits on open AI um New York Times and OpenAI was having some sort of a um sort of a legal um you know conflict going on and so on. So I think that has led to people thinking about what does copyright mean in the age of AI right. So I know there is the the you know in Hollywood there is the writer's guild that has done you know strikes and and has uh um also taken legal action against some of these companies and so on.

So um we are seeing some of that but I think in terms of regulation um we are a little bit behind um definitely much be more behind than Europe right so and this is not just for copyright even for privacy right so u so you Europe has a more robust uh privacy regulation right the GDPR uh which has guidelines such as you know um In the day a person wants their data to be you know not used by any model these uh model developers need to provide a provision for that

right so now that raises a bunch of different technical questions it's like how do you kind of delete a concept from a trained model right so that's the area of concept erasure um also called machine unlearning and uh um so those

are all becoming really important topics now to research right and and develop techniques that allow you to do that

um because of the regulation right so this is you know the regulation has now incited that these are you know important things to look at and enable

whereas in US

that's a unlearning thing

so yeah you know not being an expert in large language models um my philos my approach to understanding large language models you just keep throwing more data at it and and telling it to to to tune just getting bigger and bigger bigger data sets.

Um and it kind of just has a will of its own almost but you're now saying that we can intercede and kind of guide it a little bit right with the untraining

uh yes so I think uh um we we are still trying to develop kind of effective methods to unlearn

specific concepts without forgetting everything else that it has learned right so if I want it to to delete one specific thing I don't want it to forget everything else cuz then the model is just useless and all the hours that you spend training all the hours and resources that we spend training it are now wasted right so um so there's kind of like doing it in a balanced way where we are uh retaining the learning but you know still being

able to remove certain concepts that are being represented in the model

so

yeah um I think in US there are a few states which have privacy um acts and laws,

but not I don't think they have been updated to incorporate the newer technology.

Yeah. Well, I'm not surprised to hear you say those things that you just said, which is that Europe's taking the lead um with regulations and they seem to be unified across different countries and languages. Uh the US uh is lagging behind and it's taken up sort of patchwork between states. It's almost like Europe seems to be able to get their act together quicker uh as multiple countries than than the US as a single country.

Yeah.

But I think there are they are setting precedences. So there is a recent Supreme Court ruling saying that you know AI generated content cannot be legally copyrighted. So they are uh they are like you know there's no like unified law or anything

but I think they are thinking about it right and and I think there is a a balance between innovation and protection that I think is is the key problem here right so they

there is it is a difficult problem to solve but I think that the dialogue can be bigger I mean even if they are discussing it I think it is it deserves a lot more time and attention.

I think so, too. And that's because I think, well, here's my view.

I think that's because we all feel it, right? I mean, every single person that I've ever spoken to feels it when they see some content which is fake. It's jarring. It's um um you feel like you're being cheated um and uh lied to. Uh and that that I think that way affects you emotionally like it sort of it

somehow we have a very deep sense of wanting to know that something's genuine.

I think you you have

you have hit a very kind of uh you know important point here right cuz I don't know anybody who likes getting lied to right so I think uh

I think there is this kind of human tendency to to go after the truth it is not just for scientists I think it is even I think everybody has a mini scientist in them so maybe it's that Well said.

So uh yeah so I think uh it you know everybody has this this tendency to go after the truth right so

um whether you choose to you know deliberately deceive someone at any point in your life or not

you don't want you know some sort of deception at any level right because that is very jarring right that is uh we all like to be grounded in some way and you know this idea of of believing something that is untrue is you know extremely like it's contrary to our survival instincts right we want to know if something is coming we want to know right and we want to know it like you know in a grounded way we want proof for it and things like that

that's a very good point I always like to

when I think about something that affects me emotionally I always try to think about how it might have happened when I was in the jungle right in a tribe in the jungle or you a troop of of of chimps or something, you know, as you just said that that you want to know that that thing is a real threat over there versus

Yeah.

versus it's somebody lying to you that it's a threat because you need to

No. And also Yeah. And also the the I think it's it's more also about like the consequences of the decisions, right?

So if I'm making a decision, if you know something that is unreal is informing me and I'm making my decision based on that

u you know if I'm someone who is completely nonchalant about and doesn't care about their decisions I think that person wouldn't care if something is real or fake or whatever right but if you are someone who does care about you know the kind of decisions you make and the consequences of it

uh then I think you would care about you know if you're looking at something that is real or not.

Right. Right. Interesting insight. I hadn't thought of it that way. I think Yes. So, I mean, you could make career decisions.

Um, you've obviously committed a lot of your life and energy to getting an education in a certain direction. You want to know that that's going to have a long-term future for you. So, you would rely on truth. Are people telling you the truth for that? Good point.

Yeah. But even small decisions such as renting an apartment, right? So, say for example, you're looking at these online pictures, they all look great. Uh, you know, and not always you're able to physically travel there and look at the, you know, the property and things like that.

And say you're making these decisions online and you know, uh, you think that everything looks realistic, but then you sign this lease and now you're stuck because the apartment looks nothing like, you know, what, uh, what was shown. I have I give this example to students about dating apps, right? So I was like, oh, you know, have you ever been kind of uh um you know the person edited the picture heavily and you go and meet the person and then they are nothing like that and you spend your days you know of effort and time

and energy on this. So so I think even in our day-to-day uh activities right it is extremely important u but also for the bigger things right.

So uh who do we vote for right? If I'm if I'm consuming content to inform myself

and uh say that content is untrustworthy right but it is changing my perception of things and it is you know changing the kind of decisions I'm going to make in the future and so on. So yeah.

Yeah. Yeah. Yeah. Really important uh really important topic. I think what to sum up that I'm going to move on to something else that's of interest to both of us. Um to sum up, I think we we have this um battle going on between the generated misconcept mis misdirection and trying to keep things genuine.

Um what I think I draw some faith from is the fact that people all want to see the truth and nobody wants to be lied to. And I think that's a very powerful force and I think that will emerge. I don't think we'll get, you know, I think we won't I won't we won't get the internet full of fake stuff. Nobody wants that. Uh and I think that will have its day and but we'll see this constant battle going on. Um now let's go to some of the other areas you're working in which was the machine learning AI vision stuff which we're working on.

So this is more not so much about autogeneration um but more about deterministic and by by that I mean you're looking at a scene and you're trying to figure something out right. Yes. So, uh this is related to more recognition, right? So, uh I think uh so the some of the projects that I'm working on and one is uh with you on is this kind of uh remote well-being, right? So, the underlying research that that is interesting here is that so machine learning has done really well with objective visual tasks, right? So you show an image to a model or a person and and you know they can objectively say whether there is a car in it or not. Right?

So that is what we would call an objective visual task where if you ask five people

the chances of you getting the same answer is really high. Right? So they will all say yes or no there is a car in the image or not.

Um but there are more complex questions right more complex visual questions such as for example uh do I trust this colleague enough to be collaborating with them on a project right um

uh do I feel comfortable in this office environment right am I you know um is this kind of like a a positive environment where everybody is uh is collegial and collaborative and things like that right so whenever We are talking about these kind of human human interaction or human computer interaction or human robot interaction settings. There are more complex visual questions than just is that like a car or is this an object that is a chair or a table and so on,

right?

So, so we wanted to kind of start thinking about some of those, you know, some of the operational settings where these kinds of subjective visual tasks show up. And one of them is this kind of well-being, right? So well-being, determination.

Yes.

So um I want to be able to kind of, you know, maybe get nudges and help if I'm looking like, oh, I'm super tired and dehydrated and things like that, right?

Um a lot of the times you'll see that uh medical professionals even though the the whole online and tele health system has gotten really good and they can implement it but they will still ask you to come in person

right and one of the reasons is because there is a lot that they can judge and discern just by looking at you right and like how good are you feeling and like how you know so um so I think that is something that holds some value so we started thinking that Okay.

So, so the maybe we can have a vision technology that analyzes some of these kind of external uh appearances and you know um and try to see if we have any level of reliable signal to uh evaluate and understand the well-being of a human, right? Whether it is dehydration, stress, fatigue, things like that.

Fantastic. So, this could be useful for, for example, right now we could have some kind of monitor that's running that would tell you and tell me how how I'm feeling that day. Um, you know, maybe I haven't had enough coffees this morning or maybe I didn't sleep well or I dehydrated and I could have some sort of I guess we're all trying to determine the action to take, right?

So,

um, you know, I need to go and have a glass, a couple of glasses of water or some electrolytes or just go for a walk or or maybe there's an indicator of some other more serious uh um health issue that's emerging. I mean, certainly trying to catch something early is a great way to to fight off a disease, right? So, can we go that far with this technology?

Yes. And that is the intent, right? The intent is that at some point we'll be able to create these meaningful interventions, right? And we would kind of give it to them, you know, before something bad really happens, right? So um so I think that is what the goal is.

Um we want to be kind of uh uh getting to a point where we can easily um detect and in some cases also predict right like what is coming um next

so that we can then kind of issue good interventions right uh and in extreme cases maybe alert uh some family members or like you know uh somebody from your close circle. Yeah.

Yeah. Exactly. Um how we have a keen interest in this uh in my company and that's why we're engaged in the uh this research activity because we've been developing you know camera systems for a long time. Um we see the opportunity um to go you know to go and make this a commercial success as well as you know I guess provide some real value to people.

Um it might be as simple as you know running software on on the computer that we have right here. Right. Right.

Yes. So far, yeah, so far we are using that as uh itself. But you know, you can also think about having like a specialized, you know, camera device which has its own processor and that really kind of uh does give you the the power to do it on device, right? And where you don't have to transmit the data to another server which can cause different privacy and security problems. So I think there are like there's a lot that can be done in this space,

right?

And and I think just to illustrate to the audience the difference between doing it on device and and not is that if you can do the processing on the device that that is on the camera right there in front of you in your own home or wherever. Um you don't have to transmit the media back to the cloud. And I mean how many data breaches have there been uh or nefarious acts of you know getting access to people's data? There might be something sensitive in there that you don't want uh you don't want a video

in the cloud of you, you know, in front of the the TV at night or whatever, you know.

Yeah.

But it's also like, you know, overall, uh, companies would, you know, would benefit more from not like selling their data of their users to third party and things like that, right? So,

so if it is something that uh you know they can keep it in house, I think the customers or the users would probably trust that more

than if they are sharing that data you know because then it is very hard to trace as to who has it and what they are doing with it. So

yeah. Yeah.

And I think the other sort of point of interest is that we noticed that there's more and more people that are I guess home um working from home um but also maybe living alone at home maybe getting older and living at home alone and that I think that's expanding and also just the costs of moving somebody from their home to the the the medical practitioners or to the hospital is enormous. Um but there's a driving force in cost and in sort of just the the movement to towards what I guess is telly health.

They call this this teley health where you can kind of get some kind of health support um remotely where you actually are right now without having to make that journey and I guess you can't as you said you can't do everything but we can certainly move towards that and maybe a lot of really good services can be provided off the back of this technology to really help people. Absolutely. Yes. I think we need to start taking steps to get there. Yes.

Yeah. Yeah. So, how did you let's let's change topics back to you again, Aana. How how did you get into this field and and what are your qualifications? We didn't really cover off on that.

Yeah.

So, um I think uh I I started my undergraduate in computer science and engineering uh back at IIT Delhi in India. Um and uh I I liked it but I wasn't like you know this it didn't feel like there was a lot to it that was my calling or anything.

Uh but I I do remember taking this kind of image analysis course and uh and suddenly I had this kind of you know renewed interest in like everything that I was doing and I

uh I do think one of the reasons was that I'm a very visual person right so uh I think uh writing some program and then looking at like you know what is the impact of it visually was definitely something that uh that you know grabbed my interest in this area and uh I took more and more courses uh in this field and I think I had a specialization in image analysis and machine intelligence even as an undergrad.

So um which which was you know pretty uh something that not every undergraduate does but I went kind of very deep uh while uh you know during those years I started doing research with some of my faculty members uh and I thoroughly enjoyed it. It was one of the most rewarding experiences of my undergraduate time and u and that led me to you know think that oh this is something I can you know see myself doing long term. M

so I applied for a PhD um at various places in the US and I selected to go to University of Notre Dame

um where I got my PhD uh in computer science and engineering and uh I was a part of the computer vision research lab there.

I graduated in 2020 and uh since then I have been a professor and now I have a lab that then we try to kind of all together solve various problems in this field.

Yeah. Well, that's a great journey and um being a I guess being a specialist in in an area like this is is I mean you're in a hot field right now because not only is we talking about you know the computers being able to generate images and do that the the determination determination but what I've witnessed over the last 30 years is the power of cameras is going up and up. When I started in the field, we had VGA images, which is 30 360 360, 000 pixels. That was the best we could do in a phone.

Now, I've heard that there's a 200 megapixel camera. And I don't I think it's been out for a little while now, too. And they're so much better, too, that the noise floor is lower, the power consumption is low, they're still just as small. So, we really do have the ability to pull high quality image data and massive amounts of it. And I think that's also, you know, there's these multiple things coming together, the AI software and the area that you're working in, the camera hardware, and I guess the third element is the the processes, the neural network processes and the GPUs, um, which does the grunt work of of turning an image into some some form of meaning.

Yes, I think you're right that this is kind of all three pillars have seen like exponential growth, right? So, uh with respect to the camera technology, um you know, AI models and then also

the resources that we need to train these models, right?

So the GPUs keep getting better and you know you can now ingest more data and train bigger models and you know have these kind of generalists in some ways like this there's a lot of there's a term that's going around nowadays it's called foundation models

u so you don't need to train these kind of separate models for each specific task you can kind you can go and you know train a really huge uh foundation model that is uh able to tackle various tasks um all right within a specific domain.

So I think uh all of this has been enabled by you know what you just mentioned as um these technologies.

So that foundation model um maybe you can explain that a little bit more but um there's also this thing called fine-tuning right and you from what I understand you take a foundation model so someone's done a lot of the work for you to create something which is a structure and then you say well actually I want it to do this task over here so you fine-tune it to do something very specific did I get that right

uh yes so you can have u you know uh a for a lot of different applications s we may not require to train you know another large model um and even if uh you know even if we can there might not be enough data in a specific area right so depends on the domain uh so for example medical

images like they're not as you know uh extensive in the availability of data as uh maybe real world scene images uh that are just available you know online

so uh so I think then how do you kind have trained these models, right? So, a lot of the times you can take the knowledge of a pre-trained general model and then you can fine-tune with a smaller set of more specialized data,

right? And then kind of uh you know change a few things about what the model is supposed to do um and uh that should give us some you know some more specialized uh specific uh tool that can be used for that application. Yes.

Yeah. Yeah. So, we've talked about the present. We've talked about the the past.

What do you think's hap going to happen in the future in this area?

Oh, this is extremely difficult to to answer. I mean, past and present is easier

to to to look at and analyze. So, I I don't think any of us have a you know, a crystal ball, right? Uh and we um we all can speculate you know based on uh what we know and what we observe.

Um

I think you might have some good insights though because of your specialization right?

Yes, I I do think that you know uh this is this technology is going to help people to some extent

right in um

in their tasks or like helping them you know maybe get away from some of the mundane tasks

that uh that they have to do as part of their you know job or their daily life. Um but I think uh there are also a lot of uh kind of consequences that are not so good

that we need to be thinking about

right so the technology is only as good as you know your usage of it.

So, so I think uh that is something that uh we as users have to be cognizant about but more than that I think we need some level of guidelines and you know

um even maybe regulations to ensure that you know this is a technology that the human race can live and thrive with. So, so I think that is something that is very hard to to kind of uh um ignore, right? So, I think this is we have seen so many cases of you know um of uh repercussions of using such technology in various fronts

and I think that is something that uh is you know um that is what I think needs to happen going forward. I don't know if that will happen going forward.

So, um I think these uh these big models and you know they are going to plateau at some point, right? So, there's only so many so much capabilities um that these models can add, right?

So I think um

I do think that the more interesting scenarios now are going to be you know physical and spatial AI you know where you have these kinds of uh uh agents and robots interacting with the real world environment right so I think this kind of uh the the information chat bots is was cool in the last 5 years but I think in the next coming five years that's where we are going to see the most amount ount of innovation is because how can we solve tasks now in real life right not just cognitive tasks

what's your opinion on uh full self-driving using using cameras only only like Tesla's trying to do

yes so I mean we all would love to have that right I mean we all know it would be a

useful technology but it is an extremely hard problem to solve as we have already seen right there were a lot of predictions that by you 2015 or 2020 there will be you know self-driving cars on you know all kinds of roads but that's not the case right we are already crossed that line and um and it is because it is an extremely difficult problem to solve right you can um I think the canonical version of the problem is maybe easy right where you don't have events that happen out of the blue out of distribution and so on so forth but the human visual system as we talked about right has evolved has been trained to be really good at navigating these surroundings, making these split sec second

decisions when an unidentified object object just kind of flies in front of you, right? And and it's very hard to kind of uh encode that level of visual intelligence in these models where they can just like, you know,

use what they already know to then react to unseen scenarios, right?

So, um yeah, we have to make them better at all these open set recognition scenarios and things like that.

So, um I do see it getting better, but I don't know if it's,

you know, I was away last week in in Las Vegas, and there's this thing called the Vegas Loop, um which is an underground tunnel done by the Boring Company, which has Teslas driving around the tunnel, and I thought, well, surely they're going to have full self-driving in there. No, every single car has a driver and there's multiple people there as well shuffling people back and forth. It was the opposite to automation. And I would have thought that my instincts would tell me that that's a very controlled environment. That would be the perfect place to get full self-driving to work.

And it's it's a boring company, Elon Musk company. It's Tesla, an Elon Musk company. It's 2026 and there's no full self-driving there.

Yes. Because I think it's it's a this uh is something that we should probably learn from right because uh there is a lot of kind of questions of responsibility

right because uh if something bad happens then it is you know extremely uh detrimental to the person right so I think it is very important to to understand uh you know the responsible use of technology even though I'm pretty sure that to allow to 60 to 70% of the times.

Uh if you put it on autodriving mode, it's fine. But the risk that you're taking for the rest

is probably not worth.

Yes.

Doing the the I think so I think that is something that this this like you know balancing tradeoffs is the problem.

It is. I think that's a very good way to say it is the balancing trade-off. I mean, if there's a bad driver at night, tired, had some, uh, you know, some substances,

uh, they're not going to drive very well. Uh, and I think that the full self-driving would probably do a better job than that. But then you have a very good driver who's sharp and alert and on top of everything. Um, and some surprising event occurs, he responds very quick, he or she responds very quickly, will do a better job than full self-driving. Um so maybe we pick and choose when we use it.

I don't know.

Yeah. Yeah. I think uh you know when you have an environment that is pretty clean and constrained and things like that, right? Or you are really tired and you know that you know the machine has better capabilities at this point than you. But I would still say that like you know the the best solution would be for you for the car and you to take a break. But you know because you know this is an extremely dangerous thing to risk. But in case you are not well and you want to get to the hospital, right, that is a scenario where I would say that okay, please use it, you know, because you're talking about like, you know, lives. Yeah.

Yeah. Well, we're running out of time.

Apart, do you have anything you want to say or comments or questions or topics to cover?

So, so I think uh I I would be very interested in knowing your view of the future, right? So I think uh um you know you have also worked in this space for some time and you're looking at more and more companies and startups coming up after this AI revolution. So tell me a little bit about what do you think?

Well I think the trend is going to continue Aana.

I think uh the three pillars that we talked about which is the improving image quality coming directly from the cameras which is pixels plus other factors uh including multiple cameras and multiple sensors and even going into multisspectral stuff right where you going beyond the human vision to sense the world I think we're going to see the continuation of the the power of neural networks going onto the edge into our cell phones and and doing that high level of processing power and of course the capability of the models which is I think that trajectory is not going to slow down. I think then what does that mean in terms of applications?

And I think we're going to just see more and more intelligent applications. So, um being able to assess um complex scenes like I was talking about the towers, you know, tower cranes and construction sites, very complex scenes which could be analyzed with AI and draw meaning from that. I think that's uh where we're going to go. Cameras are cheap. They're going to be abundant. They're going to be uh very easily deployed and and then they can be aggregated into, you know, massive amounts of intelligence and draw meaning from that. I think that's that's where I think it's going to go. I don't think there's any slowdown. Um I think the on the other side, we talked about the generative side.

I kind of agree with you. There's this kind of battle going on. I think we're going to become more aware and and it's going to become a deeper understanding that we all have that we now have to sort of do things a certain way differently to how we used to and that always happens with new technology. Um we have to learn how to use it and that rolls out and so we get smarter on on how to overcome the people that try to mislead us. So that's probably a couple of things I've got there.

Yep. Thank you.

Thank you for sharing it and thank you for you know inviting me and I had a great time uh talking about this cuz you know it is something that I am definitely interested in but

but having these some sort of like highlevel questions you know and discuss discussions from time to time is definitely very helpful.

Yeah, I've been chasing you down for a while award. Um so we have to end on the key question and that is AI is it hype or help?

So uh I am going to give you know a more balanced answer. I think some of it is definitely hype

but I do think that uh generally it is going to be of more help.

Uh I we are already seeing a lot of progress in you know understanding um medical images right in in trying to understand various different uh contexts right getting information quickly and in a summarized way. Now of course there are challenges there where you know some of it is incorrect and sometimes it hallucinates and we are all going to you know work towards fixing some of those problems. Um but it can be helpful uh in doing some of these tasks. Um the hype is where you know when people start saying it's perfect right or it can solve all these extremely complex problems and it can replace people and things like that.

So I think those are the ones that like you know um like blanket predictions and you know um statements where everything is like amazing right those are the ones that I think are more from for on the hype side

right so uh we do think these models are extremely capable right

but am I going to say these models don't make mistakes no right I think we all know that they do make mistakes and even with our own interaction you don't get the same level of satisfaction with every response or every you know output that you see and so on.

So I think that is something to be extremely aware of right is that use it when it is helpful and you know but you should also use your own uh kind of decision-m powers and things like that

to know when it is wrong and I think that is what I tell the students as well right that I'm not saying like this is bad but you need to have an understanding right to a level where you can discern when it is good or when it is bad so yeah

well said, I think the and it's nice to end on that that point about the human is still in the loop.

We we're not I sometimes and didn't want to take the conversation this direction at the end here, but I do sometimes feel that people think that we won't be doing anything and and our lives will be in the hands of AI. I'm not seeing that. I'm seeing very much as humans evolving with this um and being a part of it and tied to it and just uh growing with it.

Yeah, I think there's this kind of like you know u augmentation that I foresee happening right so uh maybe you can call it as like AI augmented humans or something like that right but uh

u but uh but I think that is what we are seeing right now right we've seen people using this in their workflow and so on right and getting um more productive and getting better results and so on right so u so that's what we are seeing And I I don't think that uh that human level expertise you know is is is not valuable anymore or something like that. I don't think that is the case

because you know we are the ones guiding these um these technologies and we get to decide and create and consume. Yes.

Exactly.

All right Aana this has been a real pleasure. Thank you. Thank you very much for joining us.

Thank you so much. Thank you for the time and amazing discussion. Yes,

thank you. You too. It was fun.

Well, that was a fascinating discussion with Apana and we really got into some very interesting topics. AI, machine learning, particularly around vision systems was an area where we have a common interest and we were able to go quite deep into some interesting areas. One of the areas I think we're all aware of is the abundance of fake AI generated content. So this is not just text but it's now images and videos and it's getting better and better.

There's a famous if you like Will Smith video of him eating spaghetti which is actually getting better and better over time. Now this whole area of fake imaging isn't just uh static. It's dynamic and moving forward all the time. So we have uh regulators coming in to try and combat this. We have the platforms trying to combat this. But we also have um you know the generators that trying to create this content uh for getting likes are motivated by creating some outrageous content. So there's this kind of war going on or an arms race going on between these two uh opposing forces. Um where do we think it's going to emerge? Well, let's do the video.

You'll find out where we think it's going to go. Um but some of the interesting underlying technologies we got into which was actually getting characteristics from the pixel itself actually on the camera and and understanding the noise behavior and the characteristics of those pixels um and having that as a a fingerprint or a watermark to say that it was a real image generated by a real camera. An area I didn't know about um but is emerging now as a way to combat some of the fake content.

We also talked about how machine learning and AI vision systems are going to be used into new areas and one of the areas that Aanaro and I are working on is in detecting a person's health, their emotion and other factors as they sit in front of a camera, a regular camera or even an advanced camera. And then we talked about where this is all going. And I think we agree that you know complex uh understanding of the real world and we talked about um you know full self-driving and where that we think that's going and where other um capabilities are emerging um through this technology. It really was a great discussion. I hope you enjoyed it.

If you want to know more about Skillion, please follow the links in the description. I hope that helps and thanks for watching.