Everyone I'm going to talk about a sixth and final challenge.

To look at many of your technical challenges.

In detail.

Haitian this is all these previous challenges.

At 8 to perform a Equestria to go study to better understand.

This challenge should be done in context of all the challenges.

And this case the boys want to fight all of them.

Call from introduction.

What is the structure between modalities representation State the first information content apologies and relevant to Dad's house.

I'll give you a few gain a deeper understanding of each semester hydrogenated in the context of a.

So this is Brianna study topic.

How to say hello in Canadian multimodal.

So one big part is that of modality biases.

Training with multimodal data sometimes which caused us to dominate dominate.

This related to the fact that different modalities have different information.

And different relevance to the top.

The one that example of this is a typical questions from from the internet.

Michael Vasquez just like what color is it.

Because of by assessment 80%.

Banana images of yellow you're so going to see that 80% of the yellow 80% of these questions are going to be answered..

Let me see a real currency has a green banana going to start answering.

Biases.

That's quite a few approaches of approaching of a cackling this discussion is by no means.

One simple approach is pretty simple but very difficult to balance.

If you see is umbrella upside down and most of the times it is yes you can also put more examples.

Umbrellas not Beyonce..

Let's book balancing Adidas and the contribution of the other possible answers.

Lowercase is to balance training.

So many ways of balancing a training objectives that you're not relying Cynthia.

Some of that the colors were bananas wrong sometimes.

But sometimes this can also cause more dire consequences if a cat is going to be gender bias.

Modality biases.

One example is that you trying to do image caption.

You see the song this this woman along with the computer.

Some of the image caption.

Attaching gender bias.

Between image image.

Are you at like I'm almost I should be right for the right reasons right when he's trading a woman sitting from a computer.

As many of these biases that I found them but still is not comprehensive.

Biases that come from you tomorrow.

So this is basically.

If you start looking at this image and text tasks can you look at what the person is carry.

It looks like the image and verify that it's a woman starts a sign of higher probability to carrying a purse instead of a preface.

I want to park it is founded.

Either method is biases just a language the person is caring.

We can also measure these biases in Aquatica..

And your father have any image place the worse than the social biases to visual information actually make them more confident in reinforcing this trend Estero..

Visa by assistant result.

Cross-modal in fractions.

Set modality biases biases that result as having a better understanding of each dimension of hydrogenation.

Another thing that mention of heterogeneity is a fact that this these can I set different.

So then how can you better measure whether our models are robust to these North apologies in are different.

The one contribution.

What's the first identify different ways that you can be.

So far he's a greedy looking at the modality specific than domain specific ways in which noise.

Contiguous.

Another type of perfection can okra.

Or if you have coordinated modalities across the time dimension.

Having all modalities missing across that chunk of.

So this paper isn't she contributed all of these tests for make the testing where the modalities are robots in the presence of his mentality specific or multimodal in perfection.

Key takeaways that no it shouldn't be surprising there's a very strong trade off.

The presence of these imperfections.

Look up security at this in some categories of approaches of making sure that your morals are more of us.

1 General way to introduce.

These imperfections during training so no me I was going to train with my modalities xnxd they're perfect.

What if I also dropped individually and make sure tomorrow is still robot in a prediction.

List of all the characters of robust data and training in the more exposure these models have please imperfections during training tomorrow.

This was taken off of methazine to information without it.

Play learning a proper translation perfect for running a joint problems ignore the problem.

And again I'm not going to hear that you can look at.

Four cases in both of us..

So that's just a glimpse of different types of heterogeneous again.

Another big step challenge is to get a better understanding of the Cross Motors.

August customer interactions the capsule connection.

Is referring to the same concept dependencies referring to the concept.

Customer interactions look.

How are the infractions the result from the connections between these elements.

Has many many definition of cause more interactions.

Again how can we take this multimodal dataset.

Quantify the presence and type of court order interaction.

The arcs that are present.

Challenge this some of some clips into possible.

One big area of work is to put upon the idea.

Are they giving cannot be decomposed into some of these you tomorrow functions acting all these individual features and there are some problems.

And the first the first option of the representation before eating a glimpse of how this can help how we can take an arbitrary function f perhaps learn through this supposed to meet with you tomorrow stop functions f and f d and looking at the residual and the residual Mew face tape measures the amount of cross-modal interactions that exist in this morning.

So is additive model is equal to the nonlinear diffusion model.

Not just not edited interaction and not modeled otherwise meal captures the overall quality of the cosmo interactions.

August 20th 3 tomorrow.

This is good this is a good sanity check for talking about this gives you a sense of the overall quality of the cosmo interactions.

What is going to go with you.

He said I'm just overall presents can we go to look at the individual crossmotor interactions.

The one inside that we have is that if.

Directions to Skechers.

I'll be on the additive interactions then the second derivative of a respect to X your first feature.

Not additive interaction.

So give me the definition.

Inspires a natural second order extension of your great-aunt approaches.

If I have a talk with such as clever as you a sticker on it.

Can I take you out of my motor.

Take a first derivative with respect to some parts of.

I've got off the question.

And then take a second derivative of the second puzzle game.

So that should then tell me the parts of the image that correspond with those parts.

And this method is actually pretty if you can take the parts of the clever tomorrow does attend to the tiny yellow shiny object.

Ask the correspondence for the words tiny yellow triangle.

I like what you can do that for me.

Add another 10 to the word birds as a second-order interaction often and likewise for three small.

Lisa ways of identifying these are corresponding cross-modal interactions very specific level.

One interesting thing is that you can also do this for all that don't have two corresponding rather complementary interactions.

You can see that you take the first step to the side and the worst.

This can also capture the second-order interaction with the movement of the speaker when saying those things.

They can also help us identify certain complementary cross-modal directions.

Is there other ways about capturing all these cost more interactions be on this one so this paper called.

She said she taking them all.

I'm taking a message to Joshua tomorrow in Port Arthur.

List any time using any.

Una Moto interpretation Mass Effect.

I think a sergeant Gonzalez is on these cross-border interactions.

Is there a site was that.

Based on these influences attraction is influence of one is much larger than the other.

That's it..

If the magnitude of both our office similar and there was a point in the same direction as a complimentary pack.

Anthem attitude is similar for the opposite direction to the concert.

That's one way I wasn't hurting these calls more interactions proposed by this approach.

And why do you think things are found.

Facts about the language.

Buddy Chester.

I just have a very cool visualization website.

It is publicly available allows you to look at videos and interactions exist in these videos.

So what is your services are mostly modality and model specific specific.

See you at the mall tomorrow Transformer this recent paper also proposes an interpretation multimodal Transformers Vision language.

She said she can take these off attention across your contact data and that can be broken down.

Play Stacy's house to you tomorrow image important.

Your language language attention without you or importance in saw your text.

Legacy that is indeed just a little.

Text importance of Surfer and riding a wave highlighter in text.

Because I go deeper and look at the other two options language version attention.

You are so this highlights the correspondence and complementary interactions across Imaginext.

So again this is cool interactive website.

You can check out you want to see around with these models and interpretation methods.

So one very big challenge sure these are some of glimpse of the approaches interpreting these car models.

Typically typically don't have a ground screw.

I'll be interpretation.

Simple ideas what ideas tomorrow simulation.

And then give it to her.

Thank you niece evidences can a human then judge for the model predict.

And this would be in line with what the model actually break.

Someone handed the model was fully Black Box they didn't do any interpretation.

I recover a simile where the model predicts.

But if your interpretation approach is completely faithful tomorrow and you should be able to recover.

Another idea was to be more practical and useful ways how to modify.

Again can you take a back off tomorrow.

Run some interpretations obtain some evidence that one tomorrow is using to learn.

Give that to a human.

This continues to fix bugs and eventually of measures of performance.

But again this minneopa challenges here tomorrow actually face.

It is a exclamation Ashley Faith with a modern receiver.

Is this one clearly actually useful in his estimation help you nuts there's a.

Hypersexuality.

I want the issues that are also disagreements so you can run the same motor on different.

Interpretation of different different different explanations so how do you actually resolve the disagreement.

All of this should be studying the context of housing that evaluate these interpretation.

Another big challenge is that of a the multimodal learning process.

You know we seen many times of hydrogenate issues.

Play bryndis together are there any learning optimization challenges.

And yesterday.

The animal modalities.

We're trying to do classic Asia from RGV video.

And white core finding a half is that.

How old is Madonna.

They learn at different rates easier to learn from any more quickly learn over..

And it take longer so you don't actually learn from that way..

And again this this one disadvantage inherit from the idea of these different modalities having different..

Is it supposed to rain. The problem but when I get a proposed was to actually keep track of how much is modality is learning from.

Better balance.

One big issue here is that instead of just training one more.

Without a PC or data.

I got you a bit slower but going rates across modalities.

What's the monument open.

She's just barely scratched the surface.

I understand the present and type across more interaction.

This too many challenges.

Qualifications to be an overarching technical challenge each of these previous five challenges in learning representations how can we understand interactions in reasoning process how can you understand the learning challenges in Trinity mall tomorrow.

I would that all of these are almost open question.

And if this light overwhelms you the idea is that is true.

Quantification is prop his overarching challenge that looked at a courthouse.

It's a lot a lot open challenges.

Christine many caramel tomorrow we start with representation and alignment.

Representation did of Catherine Herridge.

Alignment the idea of capturing the connections in Tennessee's between.

How do you catch a representation and a lineman.

K I should use that to better inform our reasoning process a structured inference ideally with interpret about Concepts and capture audio..

Sometimes you might do explicit reasoning.

To get a prediction.

I want to do accessories need to better.

Chords to better transfer Knowledge from modality to the.

Of course we don't do anything all the time it would be great if he did or said that he might just skip a representation and Alignment to whatever you and toss this weather prediction generation or transfer passwords.

They finally quantification is his overarching challenge that revisit all these people challenges and tries to get a better understanding of learning and optimization challenges in this mall tomorrow message.

Again this is just a small glimpse into the whole spectrum of Baltimore research.

Coming soon to Archive are you going to release this rain you serve a paper that goes to the more in-depth into the Saxon Army captures more stop challenges and also service more than 600 papers.

Are there relevant in this area.

If we need papers isn't you think you still have this phone.

Without core content validity available alongside lecture videos.

Set the alarm.

And you want to capture something is the fact that most murders are signs of heterogeneous and interconnected.

Thank you everyone.