add

models/LLaVA/playground/data/prompts/complex_reasoning/000_caps.txt

@@ -0,0 +1,18 @@
+A man wearing multiple neck ties making a goofy face.
+A man in a white shirt wearing very many ties.
+a man with ties on poses for a picture
+A man wearing multiple ties on his neck.
+A young man smiles while wearing several ties.
+
+tie: [0.574, 0.298, 0.752, 0.704]
+tie: [0.464, 0.339, 0.639, 0.789]
+tie: [0.349, 0.363, 0.563, 0.732]
+tie: [0.259, 0.255, 0.668, 0.805]
+person: [0.019, 0.065, 0.962, 0.988]
+person: [0.0, 0.24, 0.214, 1.0]
+tie: [0.316, 0.778, 0.443, 0.867]
+tie: [0.386, 0.707, 0.496, 0.801]
+tie: [0.251, 0.354, 0.402, 0.747]
+tie: [0.44, 0.318, 0.63, 0.397]
+tie: [0.566, 0.33, 0.655, 0.543]
+tie: [0.25, 0.409, 0.359, 0.556]

models/LLaVA/playground/data/prompts/complex_reasoning/000_conv.txt

@@ -0,0 +1,5 @@
+Question:
+What is unusual about this photo?
+===
+Answer:
+In the photo, the man is wearing a total of ten ties around his neck. This is considered unusual because it is not common for someone to wear multiple ties at once. In a typical setting, a person would wear only one tie. The man's goofy facial expression also adds to the unconventional and humorous nature of the image.

models/LLaVA/playground/data/prompts/complex_reasoning/001_caps.txt

@@ -0,0 +1,18 @@
+A group of people standing outside of a black vehicle with various luggage.
+Luggage surrounds a vehicle in an underground parking area
+People try to fit all of their luggage in an SUV.
+The sport utility vehicle is parked in the public garage, being packed for a trip
+Some people with luggage near a van that is transporting it.
+
+person: [0.681, 0.242, 0.774, 0.694]
+person: [0.63, 0.222, 0.686, 0.516]
+person: [0.444, 0.233, 0.487, 0.34]
+backpack: [0.384, 0.696, 0.485, 0.914]
+backpack: [0.755, 0.413, 0.846, 0.692]
+suitcase: [0.758, 0.413, 0.845, 0.69]
+suitcase: [0.1, 0.497, 0.173, 0.579]
+bicycle: [0.282, 0.363, 0.327, 0.442]
+car: [0.786, 0.25, 0.848, 0.322]
+car: [0.783, 0.27, 0.827, 0.335]
+car: [0.86, 0.254, 0.891, 0.3]
+car: [0.261, 0.101, 0.787, 0.626]

models/LLaVA/playground/data/prompts/complex_reasoning/001_conv.txt

@@ -0,0 +1,5 @@
+Question:
+What challenges do these people face?
+===
+Answer:
+In the image, a group of people is standing outside a black SUV in a parking area, surrounded by various pieces of luggage, including suitcases and backpacks. They are facing the challenge of fitting all their luggage into the black SUV. There are multiple suitcases and backpacks to be packed, which suggests that the group has a significant amount of belongings to accommodate. They might have to strategize and arrange the luggage efficiently to ensure that everything fits properly into the vehicle. Additionally, they need to consider the comfort of the passengers and visibility while driving, so the placement of the luggage must not obstruct the driver's view or make the passengers uncomfortable during the trip.

models/LLaVA/playground/data/prompts/complex_reasoning/002_caps.txt

@@ -0,0 +1,7 @@
+There is a movie theater that displays the show times above the doors.
+A red fire hydrant is deep in the snow.
+The fire hydrant is in the snow near a recently plowed sidewalk.
+This city has had a very hard winter with snow.
+A hotel for dogs in the snow in winter.
+
+fire hydrant: [0.326, 0.612, 0.426, 0.72]

models/LLaVA/playground/data/prompts/complex_reasoning/002_conv.txt

@@ -0,0 +1,5 @@
+Question:
+What challenges might this city face?
+===
+Answer:
+The city faces challenges due to the harsh winter conditions and heavy snowfall. In the image, a red fire hydrant is almost buried deep in the snow, which indicates the significant amount of snow the city has experienced. This can lead to various challenges such as difficulties in transportation, increased risk of accidents, and disruptions to daily life. For example, the recently plowed sidewalk near the fire hydrant shows that the city has to constantly clear snow from roads and sidewalks to maintain access and safety for pedestrians and vehicles. Moreover, emergency services, like firefighters, might face challenges accessing crucial equipment, such as fire hydrants, during emergencies due to the snow accumulation. This highlights the importance of effective snow management strategies and preparedness in such cities to minimize the impact of harsh winter conditions on residents and essential services.

models/LLaVA/playground/data/prompts/complex_reasoning/system_message.txt

@@ -0,0 +1,10 @@
+You are an AI visual assistant that can analyze a single image. You receive five sentences, each describing the same image you are observing. In addition, specific object locations within the image are given, along with detailed coordinates. These coordinates are in the form of bounding boxes, represented as (x1, y1, x2, y2) with floating numbers ranging from 0 to 1. These values correspond to the top left x, top left y, bottom right x, and bottom right y.
+
+The task is to use the provided caption and bounding box information, create a plausible question about the image, and provide the answer in detail.
+
+Create complex questions beyond describing the scene.
+To answer such questions, one should require first understanding the visual content, then based on the background knowledge or reasoning, either explain why the things are happening that way, or provide guides and help to user's request.  Make the question challenging by not including the visual content details in the question so that the user needs to reason about that first.
+
+Instead of directly mentioning the bounding box coordinates, utilize this data to explain the scene using natural language. Include details like object counts, position of the objects, relative position between the objects.  
+
+When using the information from the caption and coordinates, directly explain the scene, and do not mention that the information source is the caption or the bounding box.  Always answer as if you are directly looking at the image.