Mouse Picking

This information is about how to get the objects a user pick in a scene.There are two ways provided by UE3 to get the picked objects: Hit Proxy and ray tracing. Hit Proxy works like OpenGL's selection mode. It is accurate but more expensive. Ray tracing is not that accurate because the collision shapes of objects may be not identical to their graphical shapes. However, you need a source code license to use Hit Proxy, so I would like to just talk about the ray tracing method here.

To cast a ray in a scene and see what is hit, you can use the function Actor:Trace(). It returns the first hit Actor and other hit information like hit location and hit normal. Notes it only hit Actors whose bCollideActors is enabled and ignore the Actor which the invocation is on.

class Actor extends Object
    ...

/**
 * Trace a line and see what it collides with first.
 * Takes this actor's collision properties into account.
 * Returns first hit actor, Level if hit level, or None if hit nothing.
 */
native(277) noexport final function Actor Trace
(
 out vector     HitLocation,
 out vector     HitNormal,
 vector      TraceEnd,
 optional vector    TraceStart,
 optional bool    bTraceActors,
 optional vector    Extent,
 optional out TraceHitInfo HitInfo,
 optional int    ExtraTraceFlags
);

However, we need to determine where the end points of the picking ray are. The class LocalPlayer has a function called DeProject() which can do you a favor. It can inversely transforms a position on screen to a ray in the scene. Notes the input screen position needs to be normalized.

class LocalPlayer extends Player
    ...

/** transforms 2D screen coordinates into a 3D world-space origin and direction
 * @note: use the Canvas version where possible as it already has the necessary information,
 * whereas this function must gather it and is therefore slower
 * @param ScreenPos - relative screen coordinates (0 to 1, relative to this player's viewport region)
 * @param WorldOrigin (out) - world-space origin vector
 * @param WorldDirection (out) - world-space direction vector
 */
native final function DeProject(vector2D RelativeScreenPos, out vector WorldOrigin, out vector WorldDirection);

The following code is a snippet show you how to get the picked objects. It is in a GameViewportClient thus the input won't be captured by other systems.

class UTGameViewportClient extends UDKGameViewportClient
    ...
    
var transient Actor SelectedActor;

function bool OnInputKey( int ControllerId, name Key, EInputEvent EventType, float AmountDepressed, optional bool bGamepad )
{
    local LocalPlayer LP;
    local Vector2D MousePos, ScreenSize, ScreenPos;
    local Vector WorldOrigin, WorldDir;
    local Vector HitLocation, HitNormal;
    local Vector TraceEnd;
    
    if( Key == 'LeftMouseButton' && EventType == IE_Pressed && class'Engine'.static.GetEngine().GamePlayers.Length > 0 )
    {
        LP = class'Engine'.static.GetEngine().GamePlayers[0];
        
        if( LP.Actor.IsPaused() && ! LP.Actor.myHUD.IsA('UTEntryHUD') )
        {
            MousePos = GetMousePosition();
            GetViewportSize( ScreenSize );
            
            ScreenPos.X = MousePos.X / ScreenSize.X;
            ScreenPos.Y = MousePos.Y / ScreenSize.Y;
            LP.DeProject( ScreenPos, WorldOrigin, WorldDir );
            
            TraceEnd = WorldOrigin + WorldDir * 10000;
            SelectedActor = LP.Actor.Trace( HitLocation, HitNormal, TraceEnd, WorldOrigin, true );
            
            if( SelectedActor != none )
            {
                return true;
            }
        }
    }
    
    return false;
}

exec function EditSelection()
{
    if( SelectedActor != none )
    {
        ConsoleCommand( "EDITACTOR NAME=" $ SelectedActor.Name );
    }
}

defaultproperties
{
    ...
    HandleInputKey = OnInputKey
}

The picked object is stored in the variable SelectedActor. You can replace it with an array and implement a multi-selection feature if you'd like some practice. To demonstrate it, I add a console command "EditSelection" which allows you to edit the picked object.

However, UT only shows the cursor in game when the pause menu shows up, but in editor it will also close PIE. To test the above code in editor, I would like to remove the command of closing PIE in "DefaultInput.ini".

;.Bindings=(Name="GBA_ShowMenu",Command="CloseEditorViewport | onrelease ShowMenu")
.Bindings=(Name="GBA_ShowMenu",Command="onrelease ShowMenu")

To make it easier to see which object is picked, I add the following code in HUD to show the picked Actor.

class UTGFxHudWrapper extends UTHUDBase;
   ...

event PostRender()
{
    ...
    
    DisplaySelection();
}

function DisplaySelection()
{
    local Actor SelActor;
    
    if( ShouldDisplayDebug('Selection') )
    {
        SelActor = UTGameViewportClient(class'Engine'.static.GetEngine().GameViewport).SelectedActor;
        
        Canvas.Font = GetFontSizeIndex(0);
        Canvas.SetPos( Canvas.SizeX*0.5, 0 );
        
        if( SelActor != none )
            Canvas.SetDrawColor(255, 255, 0);
        else
            Canvas.SetDrawColor(200, 200, 200);
            
        Canvas.DrawText("SELECT:" @ SelActor);
    }
}

With all the fore-mentioned modification,  now you can start a UTGame in editor and type in "showdebug selection" in the console, and then try to click the object in the scene and see what is shown in the debug text on the top.