The Core of Colorfinding
Finding objects can be really tedious for people to learn, hope this helps. Be aware that this tutorial is written for SRL for OSRS, not any other library. Certain constructs are the same, but the naming-scheme is that of SRL's.
- srl.FindColors
- ClusterTPA & SplitTPA
- T2DPointArray -> SortByMiddle
- T2DPointArray -> SortByIndex
- T2DPointArray -> SortBySize
- T2DPointArray -> FilterSize
- Putting it together
function TSRL.FindColors(out TPA: TPointArray; Color: TCTS2Color; Area: TBox): UInt32;This methods searches the screen in the area given for all pixels matching the given input color and returns a list of those colors, this is known as a TPA, array of TPoint, which simply means it's a list of points / coordinates.
To access each individual point in that list you have to keep in mind that the list starts at 0, and ends at High(MyList) access it like MyList[0], MyList[1], MyList[2], etc. The length (how many items it contains) is accessible by calling Length(MyList). For normal simple usage accessing individual points like this isn't needed, but it's important to know and understand these basics.
If you look to the right, and see how it gathers all the blue pixels in the image, now imagine what happens if there are two mario's in that animation, you'd have a single list containing the first Mario's pants and the second Mario's pants all jumbled up together. Later we are gonna talk about how we separate the two pants, so that they are in each their list.
Related
function CTS2(Color, Tolerance: Int32; HueMod, SatMod: Extended = 0.2): TCTS2Color;function Box(X1, Y1, X2, Y2: Int32): TBox;TBox = record X1,Y1, X2,Y2: Int32 end;TPoint = record X,Y: Int32 end;
out indicates that this is a Result parameter, which means that the function returns something here.
This function returns the list of points that was gathered, see animation, in the bottom of it you see it builds a list of two dimensional points (coordinates). This list is what's returned here.
The parameter expects a "CTS2" color. There's a function in SRL that's named CTS2 which you can use to build such a color. The function CTS2 expects at minimum two parameter, Color and Tolerance, but accept two additional parameters which gives you better precision when searching for colors, these two extra parameters are HueMod and SatMod.
myColor := CTS2($54323, 15);
myAccurateColor := CTS2($54323, 15, 0.754, 0.094);First you need to understand that a TBox consists of two coordinates, an upper-left coordinate, and a lower-right coordinate. This box explains where on the screen you wish to search.
To create such a structure we use a function named Box (see related definitions above)
myBox := Box(10,10, 100, 100);The result of this function is the length of the list, same as Length(myList).
{$I SRL/OSR.simba}
var
TPA: TPointArray; //the list in which we store the points
begin
if srl.FindColors(TPA, CTS2(1234567, 10), Box(4,4, 503, 336)) > 0 then
WriteLn('I found ',Length(TPA),' points');
end;[CENTER]Animation, where we search for blue colors (his pants).[/CENTER]
function TPointArray.Cluster(Dist: Int32): T2DPointArray;function TPointArray.Cluster(DistX, DistY: Int32): T2DPointArray;
ClusterTPA (and SplitTPA) is a density-based clustering algorithm: given a set of points, it groups together points that are closely packed together.
In SRL we wrap it under the name TPointArray.Cluster.
ClusterTPA walks through the list of points, starting with the first point in the list, all points that are close to this point is merged into a group with this point, once this point connect to no other point, it moves to the next point in the group, and checks if it connects to any point not already grouped, and so it goes on until every point in the group has been used to check if they have any "neighbors" that should be added to the group. Once that group is completed, a new group will be created from the first next ungrouped point in the list, where all points near it will be merged into a group with this point, etc... And so it repeat until there are no more ungrouped points in the list.
"Close" is defined by the dist parameter, ex dist=3 would mean that points that are closer than or equal to 3 pixels away are considered close enough to merge into a singular group.
ClusterTPA is particularly useful when it comes to separating objects in the game. Say you search for a rock to mine, as always you will first need to search for colors matching the color of a rock, however there are many rocks, so you will get a singular list containing all the coordinates that had the color of that rock. Now to separate each rock into each their list you need to somehow split them into each their list, this is where ClusterTPA & SplitTPA comes into the picture, it place each of the rocks in each their group, so long as there is nothing connecting the rocks (so they visually form a single object).
For reference ATPA / T2DPointArray is defined as array of TPointArray.
{$I SRL/OSR.simba}
var
TPA: TPointArray; //the list in which we store the points
ATPA: T2DPointArray; //the groups we found
begin
if srl.FindColors(TPA, CTS2(1234567, 10), Box(4,4, 503, 336)) > 0 then
begin
WriteLn('I found ',Length(TPA),' points');
ATPA := TPA.Cluster(4);
WriteLn('I split the points into ',Length(ATPA),' groups');
end;
end;procedure T2DPointArray.SortByMiddle(From: TPoint);
Alright, so now we have a ton of groups, in order to click the group closest to our-self we change the order in which the groups are stored, so that the group closest to us is the first one in the list of groups. This is where T2DPointArray.SortByMiddle comes to play. It will reorder the groups, so that the first group in the list is closest to the given point, which in our case would be the center of the mainscreen, the second group is the second closest to us, and so on.
However, this method works based of the mean / geometric middle of the group, so if you have a circle like in the animation the middle of the circle, well that's the center of the circle, so even tho the circle is far away from the center of the image it will be considered very close to the center, see animation.
{$I SRL/OSR.simba}
var
TPA: TPointArray; //the list in which we store the points
ATPA: T2DPointArray; //the groups we found
begin
if srl.FindColors(TPA, CTS2(1234567, 10), Box(4,4, 503, 336)) > 0 then
begin
WriteLn('I found ',Length(TPA),' points');
ATPA := TPA.Cluster(4);
WriteLn('I split the points into ',Length(ATPA),' groups');
// Make the first group in the list of groups `ATPA` closest to us.
ATPA.SortByMiddle(mainscreen.GetMiddle);
end;
end;procedure T2DPointArray.SortByIndex(From: TPoint; Index: Int32 = 0);
So, to avoid the above problem the method SortByIndex solves the sorting by instead of sorting by the geometric mean, it instead sorts the groups by looking at the n-th coordinate in the group, and checking how far away that's from the given point (in our case center). By default it will use the first coordinate in every group index: Int32 = 0.
{$I SRL/OSR.simba}
var
TPA: TPointArray; //the list in which we store the points
ATPA: T2DPointArray; //the groups we found
begin
if srl.FindColors(TPA, CTS2(1234567, 10), Box(4,4, 503, 336)) > 0 then
begin
WriteLn('I found ',Length(TPA),' points');
ATPA := TPA.Cluster(4);
WriteLn('I split the points into ',Length(ATPA),' groups');
// Make the first group in the list of groups `ATPA` closest to us.
ATPA.SortByIndex(mainscreen.GetMiddle);
end;
end;[CENTER]Animation of sorted by first index[/CENTER]
procedure T2DPointArray.SortBySize(Size: Int32 = 0; ClosestFirst: Boolean = False);
Another way of sorting the groups is to sort them by their individual group size / length, say you know you are searching for a really large object, you can sort the groups so that the largest groups are the first ones. This function can also be reversed, so that the smallest groups come out first in the order, however that's not nearly as often needed.
{$I SRL/OSR.simba}
var
Group,TPA: TPointArray; //the list in which we store the points
ATPA: T2DPointArray; //the groups we found
begin
if srl.FindColors(TPA, CTS2(1234567, 10), Box(4,4, 503, 336)) > 0 then
begin
WriteLn('I found ',Length(TPA),' points');
ATPA := TPA.Cluster(4);
WriteLn('I split the points into ',Length(ATPA),' groups');
// Make the first group in the list of groups `ATPA` largest one
ATPA.SortBySize();
// write the size of every group:
for group in ATPA do
WriteLn('This group has the size: ', Length(group));
end;
end;procedure T2DPointArray.FilterSize(MinLen, MaxLen: Int32); overload;
Filter out groups less than 4 points FilterSize(4,999999)
Filter out groups less than 4 and more than 60 points FilterSize(4,60)
In most cases you know that the size of the interesting groups roughly contains a certain number of points, so this is where ATPA.FilterSize helps out, it will remove outlier groups that does not have a length within the given range.
So for example if your list of groups are the following lengths / sizes:
- 66, 45, 9, 1, 54, 4265
And you know that the object you are searching for roughly has 50 pixels.. more or less, you want to filter out everything notably less than that, and notably more than that.
ATPA.FilterSize(30, 90)
Now we have removed everything smaller than 30, and every group that's larger than 90 so what you are left with is:
- 66, 45, 54
{$I SRL/OSR.simba}
var
TPA: TPointArray; //the list in which we store the points
ATPA: T2DPointArray; //the groups we found
begin
if srl.FindColors(TPA, CTS2(1234567, 10), Box(4,4, 503, 336)) > 0 then
begin
WriteLn('I found ',Length(TPA),' points');
ATPA := TPA.Cluster(4);
WriteLn('I split the points into ',Length(ATPA),' groups');
// filter out all groups that contains less than 200 points and more than 1000 points
ATPA.FilterSize(200,1000);
WriteLn('After filtering out outliers we have ',Length(ATPA),' groups left');
// Make the first group in the list of groups `ATPA` closest to us.
ATPA.SortByIndex(mainscreen.GetMiddle);
end;
end;So by mainly using the methods we have learnt about so far we can write a small object finder. Keep in mind that color, tolerance and uptext-check, and the FilterSize parameters in the following code are all placeholders, you'd need to change those to fit whatever you wanna search for.
var
TPA,Group: TPointArray; //the list in which we store the points
ATPA: T2DPointArray; //the groups we found
i: Int32;
begin
if srl.FindColors(TPA, CTS2(1234567, 10), Mainscreen.GetBounds) > 0 then
begin
WriteLn('I found ',Length(TPA),' points');
ATPA := TPA.Cluster(4);
WriteLn('I split the points into ',Length(ATPA),' groups');
// remove none-interesting outliers so we don't have to mouse over them
ATPA.FilterSize(250, 1500);
// Make the first group in the list of groups `ATPA` closest to us.
ATPA.SortByIndex(mainscreen.GetMiddle);
// loop through every group, nearest to furthest away
for Group in ATPA do
begin
// move mouse to a random point in the group
// We access a random individual point in that group by indexing
//
// Breaking it down:
// Length(Group) returns the number of points in the group
// Random( .. ) generates a random value in the range 0..Length(Group)-1
// Group[ .. ] accesses the item at that random index.
Mouse.Move( Group[Random(Length(Group))] );
// check if the uptext matches our expectations
if MainScreen.IsUpText('Change me') then
begin
// click this object!!
Mouse.Click(MOUSE_LEFT);
if not MainScreen.DidRedClick() then
break; // we should probably return False here, as we missed clicked.
// stop the loop, we found what we were looking for!
Break;
end;
// we end up here if the group didn't match our expectations
// so we will move to the next group and see if that's any better.
end;
end;
end;Final notice:
- As I find time I can probably document and create animations for more of the core methods.
- Don't worry too much about it if you come across fucked up of horribly confusing sentences, that's probably just a result of me writing all of this at once, without a break.
- Animations are made in Simba, and then recorded with a screen recording tool.
- For a smaller less detailed related tutorial see Find objects with SRL & RSWalker.
- For setting up Simba for SRL and RSWalker have a look at Setting up SRL with RSWalker or Complete Guide to Setting Up Simba and SRL.