feat: initialize Streamingle Utilities package

This commit is contained in:
KINDNICK 2026-07-23 22:42:49 +09:00
commit e9bbf8481a
107 changed files with 9746 additions and 0 deletions

1
.gitattributes vendored Normal file
View File

@ -0,0 +1 @@
*.meta -whitespace

6
CHANGELOG.md Normal file
View File

@ -0,0 +1,6 @@
# Changelog
## 0.1.0 - Unreleased
- Extracted Gaze, transform constraints, SmoothFollow, Timeline subtitles, and animation cleanup into a single UPM package.
- Standardized package folders and Unity menu roots under `Streamingle Utilities`.

7
CHANGELOG.md.meta Normal file
View File

@ -0,0 +1,7 @@
fileFormatVersion: 2
guid: 6179a67f6d26d5b43bc5c81467bc82c5
TextScriptImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

73
Documentation~/Gaze.md Normal file
View File

@ -0,0 +1,73 @@
# Blendshape Gaze Timeline
`SkinnedMeshRenderer`의 블렌드쉐입만으로 캐릭터 시선을 타깃/카메라에 고정하는 시스템입니다. 3×3로 캘리브레이션한 포즈를 yaw/pitch에 따라 보간하며, 재생 중에는 `LateUpdate`에서 기존 페이셜의 eyeLook 채널을 최종 합성합니다.
## 캐릭터 세팅
1. 캐릭터 루트에 `BlendshapeGazeDriver`를 추가합니다.
2. `Target Renderer`에 Face `SkinnedMeshRenderer`를 직접 할당합니다.
3. Driver 인스펙터에서 `Open Guided Calibration`을 누릅니다.
4. 전용 창에서 `Create Profile`, `Sync ARKit EyeLook 8`을 순서대로 누릅니다.
5. `Auto Create Gaze Origin`을 누릅니다. eyeLook 블렌드쉐입이 움직이는 좌우 정점 중심을 사용하며, 계산할 수 없으면 Renderer bounds 중심을 사용합니다.
6. `Face_GazeOrigin`은 Head 하위에 두고 `Align Origin Forward to Scene Camera`로 로컬 +Z를 정면에 맞춥니다. Gaze Origin의 월드 회전이 항상 시선 방향 기준이며, `Head Reference`는 Origin이 없을 때만 대체 기준으로 사용됩니다.
`Target Camera`를 지정하면 타깃이 비어 있는 Timeline Clip이 해당 카메라를 봅니다. 지정하지 않으면 `Camera.main`을 사용합니다.
프로필은 Sync한 Source Mesh를 함께 기억하므로 다른 의상의 Face를 잘못 연결하면 인스펙터에서 경고하고 출력을 중단합니다. 동일한 캘리브레이션을 호환 Mesh에 의도적으로 공유할 때만 Source Mesh를 비웁니다.
## 가이드 캘리브레이션
Driver 인스펙터의 `Open Guided Calibration` 또는 메뉴 `Tools > Streamingle > Gaze Calibration`로 창을 엽니다.
1. 설정 체크가 모두 정상인지 확인하고 `Start Calibration`을 누릅니다. 프로필은 저장되지 않는 작업 복사본에서 편집되고, 이때의 얼굴 값도 안전하게 보관됩니다. 종료하거나 창을 닫거나 씬/프리팹 저장을 시작하면 원래 얼굴 포즈로 복원됩니다.
2. ARKit 표준 eyeLook 8채널이면 `Auto Seed ARKit`으로 기본 포즈를 한 번에 만들 수 있습니다.
3. 패드에서 Center, Left, Right, Up, Down을 선택하고 좌·우 눈 슬라이더로 직접 다듬습니다. 방향 이름은 **캐릭터 기준**입니다.
4. `Generate Corners`로 네 모서리를 자동 생성합니다. 필요한 경우 `Advanced 3x3`을 켜고 모서리를 개별 보정합니다.
5. 9개 포인트가 준비되면 빈 공간을 드래그해 `Live Interpolation` 결과를 확인합니다.
6. 9/9면 `Apply & Finish`, 작업 중간이면 `Save Draft & Finish`로 마칩니다. 프로필은 저장되고, 캘리브레이션 전 얼굴 포즈가 복원됩니다. `Cancel & Revert` 또는 창 닫기는 이번 세션의 프로필 변경까지 되돌립니다.
`Mirror`, `Reset`, `Copy/Paste` 도구로 반복 작업을 줄일 수 있습니다. Corrective 채널은 -100~100을 직접 조절할 수 있습니다.
| Pitch / Yaw | Left (Min Yaw) | Center | Right (Max Yaw) |
| --- | --- | --- | --- |
| Up (Max Pitch) | Up Left | Up | Up Right |
| Center | Left | Center | Right |
| Down (Min Pitch) | Down Left | Down | Down Right |
9개 셀이 모두 준비되어야 런타임 출력이 활성화됩니다. 시야각 밖의 타깃은 가장자리 값으로 고정됩니다. 비대칭 시야각은 전용 창의 `Asymmetric View Limits`에서 설정합니다.
추가 눈꺼풀 보정이 필요하면 프로필 Channels에 블렌드쉐입을 추가하고 Usage를 `Corrective`로 지정한 뒤 9개 샘플을 다시 캡처합니다. 기본 `Additive From Center` 모드는 방향별 값과 Center 값의 차이만 기존 blink 위에 더하므로 페이셜 표정을 보존합니다. 필요하면 채널별로 `Override`를 선택할 수도 있습니다. Timeline Clip의 `Enable Correctives`가 켜진 구간에서만 적용됩니다.
서로 크로스페이드하는 Clip은 `Enable Correctives` 설정을 동일하게 맞추는 것을 권장합니다. 설정이 다르면 교차 구간의 우세 Clip이 바뀌는 시점에 Corrective 사용 여부도 전환됩니다.
## Timeline
1. Timeline에 `Blendshape Gaze Track`을 추가합니다.
2. Track 바인딩에 캐릭터의 `BlendshapeGazeDriver`를 할당합니다.
3. `Blendshape Gaze Clip`을 추가하고 구간과 블렌드 인/아웃을 조정합니다.
4. Clip Target을 비우면 Driver의 Target Camera 또는 Main Camera를 사용합니다. 다른 Transform을 직접 지정할 수도 있습니다.
현재 지원 구성은 `BlendshapeGazeDriver` 하나당 활성 Gaze Track 하나입니다. 여러 Gaze Track을 같은 Driver에 동시에 바인딩하면 영향도가 가장 큰 Track 하나가 선택되므로, 클립 블렌딩은 같은 Track 안에서 구성합니다.
Clip 영향도 1에서는 캘리브레이션 eyeLook 값이 기존 페이셜 값을 완전히 덮고, 블렌드 구간에서는 기존 값과 자연스럽게 보간합니다. 런타임 최종 쓰기는 `LateUpdate`에 수행되므로 Animator와 실시간 페이셜 리시버의 `Update` 출력 뒤에 적용됩니다.
에디터 Timeline 미리보기에서는 Animation Track이 Head/Gaze Origin 회전과 페이셜 블렌드쉐입을 적용한 뒤 예약된 최종 Gaze 패스를 실행합니다. 따라서 스크럽 중에도 현재 모션의 머리 방향을 기준으로 계산하고, 구성된 eyeLook 채널은 해당 프레임의 페이셜 값 위에 마지막으로 합성됩니다.
카메라 컷은 Driver의 `Cut Response``Snap` 또는 `Smooth`를 선택합니다. Clip의 `Override Cut Response`를 켜면 구간별로 다른 정책과 전환 시간을 사용할 수 있습니다. `Cut Detection Angle`이 0이면 같은 카메라 오브젝트의 포즈 점프 감지를 끄며, 타깃 오브젝트 교체는 계속 컷으로 처리됩니다.
### 독쥐 콘서트 씬
열린 `독쥐_260709_콘서트_렌더링` 씬에는 곡마다 `TimeLine/<곡 이름>/Motion` PlayableDirector가 있습니다. Gaze Track은 이 Motion Timeline에 추가합니다. Driver의 Target Renderer는 해당 곡/의상의 `Face_*`를 직접 연결합니다. 예를 들어 Promise Way는 `TimeLine/Promise Way/Character/.../Promiseway/Face_Promiseway`입니다. 다른 곡도 같은 구조의 `Face_밤하늘별`, `Face_단나사마`, `Face_오르트구름`, `Face_파도혁명`을 사용합니다.
## 실시간 확장 API
Timeline 외 입력은 같은 Driver를 그대로 사용할 수 있습니다.
```csharp
driver.SetRuntimeTarget(targetTransform, influence: 1f);
driver.SetRuntimeTargetPosition(worldPosition, influence: 1f);
driver.ClearRuntimeTarget();
```
활성 Timeline 요청이 있으면 Runtime Target보다 우선합니다. `SetRuntimeTarget(null)`은 Driver의 Target Camera를 먼저 사용하고, 지정되지 않았으면 `Camera.main`을 사용합니다.
현재 버전은 AnimationClip 베이크를 만들지 않습니다.

View File

@ -0,0 +1,7 @@
fileFormatVersion: 2
guid: a37dc1ec89f2e554b935c85b7482810e
TextScriptImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

8
Editor.meta Normal file
View File

@ -0,0 +1,8 @@
fileFormatVersion: 2
guid: b0d150a89cce7e843b7e4346cf32678f
folderAsset: yes
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

View File

@ -0,0 +1,8 @@
fileFormatVersion: 2
guid: a819c052b0bc4324b4e9c30da130729a
folderAsset: yes
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

View File

@ -0,0 +1,361 @@
using System.Collections.Generic;
using System.IO;
using System.Linq;
using UnityEditor;
using UnityEngine;
namespace Streamingle.AnimationCleanup.Editor
{
/// <summary>
/// Removes contaminated keys at the boundaries of shared blend-shape recording gaps.
/// No keys are generated and the original clip is never modified.
/// </summary>
public sealed class BlendShapeSpikeCleanupWindow : EditorWindow
{
private const float TimeEpsilon = 0.00001f;
private AnimationClip sourceClip;
private int minimumGapFrames = 6;
private int keysBeforeGap = 2;
private int keysAfterGap = 2;
private int maximumBoundaryDistanceFrames = 3;
private bool linearizeGapBridge = true;
private string summary = "애니메이션 클립을 선택하세요.";
[MenuItem("Tools/Streamingle Utilities/Animation/Blend Shape Gap Cleanup")]
private static void Open()
{
GetWindow<BlendShapeSpikeCleanupWindow>("공백 경계 튐 정리");
}
private void OnEnable()
{
minSize = new Vector2(500f, 330f);
}
private void OnGUI()
{
EditorGUILayout.LabelField("블렌드셰이프 공백 경계 튐 정리", EditorStyles.boldLabel);
EditorGUILayout.HelpBox(
"모든 블렌드셰이프 키가 함께 비는 녹화 누락 구간을 찾습니다. " +
"그 구간의 바로 앞·뒤에 남은 오염 키만 제거합니다. 새 키를 만들거나 스무스를 주지 않으며, 원본 대신 복제본을 만듭니다.",
MessageType.Info);
EditorGUI.BeginChangeCheck();
sourceClip = (AnimationClip)EditorGUILayout.ObjectField("대상 애니메이션 클립", sourceClip, typeof(AnimationClip), false);
if (EditorGUI.EndChangeCheck())
{
summary = sourceClip == null ? "애니메이션 클립을 선택하세요." : "분석 버튼을 눌러 공백 경계 키 수를 확인하세요.";
}
using (new EditorGUI.DisabledScope(sourceClip == null))
{
float frameRate = sourceClip == null ? 30f : Mathf.Max(1f, sourceClip.frameRate);
EditorGUILayout.LabelField(string.Format("클립 프레임레이트: {0:F2}", frameRate), EditorStyles.miniLabel);
minimumGapFrames = Mathf.Max(1, EditorGUILayout.IntField(new GUIContent("최소 공백 길이 (프레임)", "이 프레임 이상 모든 블렌드셰이프 키가 비는 구간만 처리합니다."), minimumGapFrames));
keysBeforeGap = Mathf.Clamp(EditorGUILayout.IntField(new GUIContent("공백 직전 제거 키 수", "공백 시작 지점에서 과거 방향으로 제거할 키 수입니다."), keysBeforeGap), 0, 8);
keysAfterGap = Mathf.Clamp(EditorGUILayout.IntField(new GUIContent("공백 직후 제거 키 수", "공백 종료 지점에서 미래 방향으로 제거할 키 수입니다."), keysAfterGap), 0, 8);
int requiredDistance = Mathf.Max(keysBeforeGap, keysAfterGap);
maximumBoundaryDistanceFrames = Mathf.Max(requiredDistance, EditorGUILayout.IntField(new GUIContent("최대 경계 거리 (프레임)", "키가 공백 경계에서 이 거리보다 멀면 안전하게 건너뜁니다."), maximumBoundaryDistanceFrames));
linearizeGapBridge = EditorGUILayout.Toggle(new GUIContent("공백 구간 부드럽게 연결", "경계 키 제거 후 남은 양끝의 탄젠트를 같은 기울기로 맞춰, 공백 구간을 오버슈트 없이 직선으로 연결합니다."), linearizeGapBridge);
EditorGUILayout.BeginHorizontal();
if (GUILayout.Button("적용 대상 분석"))
{
Analyze();
}
if (GUILayout.Button("복제본 만들기", GUILayout.Height(22f)))
{
CreateCopy();
}
EditorGUILayout.EndHorizontal();
}
EditorGUILayout.Space(8f);
EditorGUILayout.LabelField("작업 상태", EditorStyles.miniBoldLabel);
EditorGUILayout.LabelField(summary, EditorStyles.wordWrappedMiniLabel);
}
private void Analyze()
{
float frameRate = Mathf.Max(1f, sourceClip.frameRate);
List<GapRange> gaps = FindSharedBlendShapeGaps(sourceClip, minimumGapFrames / frameRate);
RemovalPlan plan = BuildPlan(sourceClip, gaps, frameRate);
summary = string.Format(
"공백 {0}개를 찾았습니다. {1}개 커브에서 경계 키 {2}개를 제거할 수 있습니다.",
gaps.Count,
plan.curveCount,
plan.keyCount);
}
private void CreateCopy()
{
string sourcePath = AssetDatabase.GetAssetPath(sourceClip);
if (string.IsNullOrEmpty(sourcePath))
{
EditorUtility.DisplayDialog("복제본 생성 불가", "프로젝트에 저장된 AnimationClip을 선택해야 합니다.", "확인");
return;
}
float frameRate = Mathf.Max(1f, sourceClip.frameRate);
List<GapRange> gaps = FindSharedBlendShapeGaps(sourceClip, minimumGapFrames / frameRate);
RemovalPlan plan = BuildPlan(sourceClip, gaps, frameRate);
if (plan.keyCount == 0)
{
summary = string.Format("공백 {0}개를 찾았지만 제거할 경계 키가 없습니다.", gaps.Count);
return;
}
string directory = Path.GetDirectoryName(sourcePath)?.Replace('\\', '/') ?? "Assets";
string destinationPath = AssetDatabase.GenerateUniqueAssetPath(directory + "/" + sourceClip.name + "_GapEdgeCleaned.anim");
if (!AssetDatabase.CopyAsset(sourcePath, destinationPath))
{
EditorUtility.DisplayDialog("복제 실패", "선택한 AnimationClip의 복제본을 만들지 못했습니다.", "확인");
return;
}
AnimationClip cleanedClip = AssetDatabase.LoadAssetAtPath<AnimationClip>(destinationPath);
int removedKeyCount = ApplyPlan(cleanedClip, plan, gaps, linearizeGapBridge);
EditorUtility.SetDirty(cleanedClip);
AssetDatabase.SaveAssets();
AssetDatabase.Refresh();
Selection.activeObject = cleanedClip;
EditorGUIUtility.PingObject(cleanedClip);
summary = string.Format(
"{0} 생성 완료: 공백 {1}개에서 {2}개 커브의 경계 키 {3}개를 제거{4}.",
Path.GetFileName(destinationPath),
gaps.Count,
plan.curveCount,
removedKeyCount,
linearizeGapBridge ? "하고 부드럽게 연결했습니다" : "했습니다");
}
private RemovalPlan BuildPlan(AnimationClip clip, List<GapRange> gaps, float frameRate)
{
Dictionary<EditorCurveBinding, List<int>> indicesByBinding = new Dictionary<EditorCurveBinding, List<int>>();
float maximumDistance = maximumBoundaryDistanceFrames / Mathf.Max(1f, frameRate);
foreach (EditorCurveBinding binding in AnimationUtility.GetCurveBindings(clip))
{
if (!IsBlendShape(binding))
{
continue;
}
AnimationCurve curve = AnimationUtility.GetEditorCurve(clip, binding);
if (curve == null || curve.length < 3)
{
continue;
}
Keyframe[] keys = curve.keys;
HashSet<int> indices = new HashSet<int>();
foreach (GapRange gap in gaps)
{
AddBeforeGapKeys(keys, indices, gap.startTime, maximumDistance);
AddAfterGapKeys(keys, indices, gap.endTime, maximumDistance);
}
if (indices.Count > 0)
{
indicesByBinding.Add(binding, indices.OrderByDescending(index => index).ToList());
}
}
return new RemovalPlan(indicesByBinding);
}
private void AddBeforeGapKeys(Keyframe[] keys, HashSet<int> indices, float startTime, float maximumDistance)
{
int index = FindLastKeyAtOrBefore(keys, startTime);
if (index < 0 || startTime - keys[index].time > maximumDistance)
{
return;
}
for (int count = 0; count < keysBeforeGap; count++, index--)
{
if (index <= 0 || startTime - keys[index].time > maximumDistance || keys.Length - indices.Count <= 2)
{
break;
}
indices.Add(index);
}
}
private void AddAfterGapKeys(Keyframe[] keys, HashSet<int> indices, float endTime, float maximumDistance)
{
int index = FindFirstKeyAtOrAfter(keys, endTime);
if (index < 0 || keys[index].time - endTime > maximumDistance)
{
return;
}
for (int count = 0; count < keysAfterGap; count++, index++)
{
if (index >= keys.Length - 1 || keys[index].time - endTime > maximumDistance || keys.Length - indices.Count <= 2)
{
break;
}
indices.Add(index);
}
}
private static int ApplyPlan(AnimationClip clip, RemovalPlan plan, List<GapRange> gaps, bool linearizeGaps)
{
int removedKeyCount = 0;
foreach (KeyValuePair<EditorCurveBinding, List<int>> entry in plan.indicesByBinding)
{
AnimationCurve curve = AnimationUtility.GetEditorCurve(clip, entry.Key);
if (curve == null)
{
continue;
}
foreach (int index in entry.Value)
{
if (index > 0 && index < curve.length - 1)
{
curve.RemoveKey(index);
removedKeyCount++;
}
}
if (linearizeGaps)
{
foreach (GapRange gap in gaps)
{
LinearizeGapBridge(curve, gap);
}
}
AnimationUtility.SetEditorCurve(clip, entry.Key, curve);
}
return removedKeyCount;
}
private static void LinearizeGapBridge(AnimationCurve curve, GapRange gap)
{
Keyframe[] keys = curve.keys;
int startIndex = FindLastKeyAtOrBefore(keys, gap.startTime);
int endIndex = FindFirstKeyAtOrAfter(keys, gap.endTime);
if (startIndex < 0 || endIndex < 0 || startIndex >= endIndex)
{
return;
}
float duration = keys[endIndex].time - keys[startIndex].time;
if (duration <= TimeEpsilon)
{
return;
}
float slope = (keys[endIndex].value - keys[startIndex].value) / duration;
AnimationUtility.SetKeyBroken(curve, startIndex, true);
AnimationUtility.SetKeyBroken(curve, endIndex, true);
AnimationUtility.SetKeyRightTangentMode(curve, startIndex, AnimationUtility.TangentMode.Free);
AnimationUtility.SetKeyLeftTangentMode(curve, endIndex, AnimationUtility.TangentMode.Free);
keys = curve.keys;
Keyframe startKey = keys[startIndex];
Keyframe endKey = keys[endIndex];
startKey.outTangent = slope;
endKey.inTangent = slope;
keys[startIndex] = startKey;
keys[endIndex] = endKey;
curve.keys = keys;
}
private static List<GapRange> FindSharedBlendShapeGaps(AnimationClip clip, float minimumGapDuration)
{
List<float> keyTimes = new List<float>();
foreach (EditorCurveBinding binding in AnimationUtility.GetCurveBindings(clip))
{
if (!IsBlendShape(binding))
{
continue;
}
AnimationCurve curve = AnimationUtility.GetEditorCurve(clip, binding);
if (curve != null)
{
keyTimes.AddRange(curve.keys.Select(key => key.time));
}
}
keyTimes.Sort();
List<GapRange> gaps = new List<GapRange>();
for (int index = 1; index < keyTimes.Count; index++)
{
float previous = keyTimes[index - 1];
float current = keyTimes[index];
if (current - previous >= minimumGapDuration)
{
gaps.Add(new GapRange(previous, current));
}
}
return gaps;
}
private static int FindLastKeyAtOrBefore(Keyframe[] keys, float time)
{
for (int index = keys.Length - 1; index >= 0; index--)
{
if (keys[index].time <= time + TimeEpsilon)
{
return index;
}
}
return -1;
}
private static int FindFirstKeyAtOrAfter(Keyframe[] keys, float time)
{
for (int index = 0; index < keys.Length; index++)
{
if (keys[index].time >= time - TimeEpsilon)
{
return index;
}
}
return -1;
}
private static bool IsBlendShape(EditorCurveBinding binding)
{
return binding.propertyName.IndexOf("blendShape.", System.StringComparison.OrdinalIgnoreCase) >= 0;
}
private readonly struct GapRange
{
public readonly float startTime;
public readonly float endTime;
public GapRange(float startTime, float endTime)
{
this.startTime = startTime;
this.endTime = endTime;
}
}
private sealed class RemovalPlan
{
public readonly Dictionary<EditorCurveBinding, List<int>> indicesByBinding;
public readonly int curveCount;
public readonly int keyCount;
public RemovalPlan(Dictionary<EditorCurveBinding, List<int>> indicesByBinding)
{
this.indicesByBinding = indicesByBinding;
curveCount = indicesByBinding.Count;
keyCount = indicesByBinding.Sum(pair => pair.Value.Count);
}
}
}
}

View File

@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 0b03e044d22c4017946616c520dbeee8
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:

8
Editor/Constraints.meta Normal file
View File

@ -0,0 +1,8 @@
fileFormatVersion: 2
guid: d03e575508fe405da6778f2c68030e3f
folderAsset: yes
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

View File

@ -0,0 +1,274 @@
using Streamingle.Utils.RoughConstraints;
using UnityEditor;
using UnityEditorInternal;
using UnityEngine;
namespace Streamingle.Utils.RoughConstraintsEditor
{
[CustomEditor(typeof(RoughConstraintBase), true)]
[CanEditMultipleObjects]
public sealed class RoughConstraintEditor : UnityEditor.Editor
{
private SerializedProperty constraintActive;
private SerializedProperty locked;
private SerializedProperty weight;
private SerializedProperty sources;
private SerializedProperty damping;
private SerializedProperty updateMode;
private SerializedProperty useUnscaledTime;
private SerializedProperty snapOnEnable;
private ReorderableList sourceList;
private bool roughFollowFoldout = true;
private bool settingsFoldout = true;
private void OnEnable()
{
constraintActive = serializedObject.FindProperty("constraintActive");
locked = serializedObject.FindProperty("locked");
weight = serializedObject.FindProperty("weight");
sources = serializedObject.FindProperty("sources");
damping = serializedObject.FindProperty("damping");
updateMode = serializedObject.FindProperty("updateMode");
useUnscaledTime = serializedObject.FindProperty("useUnscaledTime");
snapOnEnable = serializedObject.FindProperty("snapOnEnable");
sourceList = new ReorderableList(serializedObject, sources, true, true, true, true)
{
drawHeaderCallback = DrawSourceHeader,
drawElementCallback = DrawSourceElement,
elementHeightCallback = _ => EditorGUIUtility.singleLineHeight + 6f,
onAddCallback = OnAddSource
};
}
public override void OnInspectorGUI()
{
serializedObject.Update();
DrawCommonSettings();
DrawSourceList();
DrawTypeSettings();
serializedObject.ApplyModifiedProperties();
}
private void DrawCommonSettings()
{
EditorGUILayout.PropertyField(constraintActive, new GUIContent("Constraint Active"));
DrawLockField();
EditorGUILayout.Slider(weight, 0f, 1f, new GUIContent("Weight"));
EditorGUILayout.BeginHorizontal();
GUILayout.FlexibleSpace();
if (GUILayout.Button("Activate", EditorStyles.miniButton, GUILayout.Width(72f)))
{
ApplyAction("Activate Rough Constraint", ActivateConstraint);
}
if (GUILayout.Button("Zero", EditorStyles.miniButton, GUILayout.Width(56f)))
{
ZeroOffsets();
}
EditorGUILayout.EndHorizontal();
}
private void DrawLockField()
{
EditorGUI.BeginChangeCheck();
var newValue = EditorGUILayout.Toggle(new GUIContent("Lock"), locked.boolValue);
if (!EditorGUI.EndChangeCheck())
{
return;
}
serializedObject.ApplyModifiedProperties();
foreach (var selectedTarget in targets)
{
var constraint = (RoughConstraintBase)selectedTarget;
Undo.RecordObject(constraint, "Change Rough Constraint Lock");
constraint.SetLocked(newValue);
EditorUtility.SetDirty(constraint);
}
serializedObject.Update();
}
private void DrawSourceList()
{
EditorGUILayout.Space(2f);
sourceList.DoLayoutList();
EditorGUILayout.Space(4f);
}
private void DrawTypeSettings()
{
roughFollowFoldout = EditorGUILayout.Foldout(roughFollowFoldout, "Rough Follow", true);
if (roughFollowFoldout)
{
using (new EditorGUI.IndentLevelScope())
{
EditorGUILayout.PropertyField(damping, new GUIContent("Damping"));
EditorGUILayout.PropertyField(updateMode, new GUIContent("Update Mode"));
EditorGUILayout.PropertyField(useUnscaledTime, new GUIContent("Use Unscaled Time"));
EditorGUILayout.PropertyField(snapOnEnable, new GUIContent("Snap On Enable"));
}
}
settingsFoldout = EditorGUILayout.Foldout(settingsFoldout, GetSettingsTitle(), true);
if (!settingsFoldout)
{
return;
}
using (new EditorGUI.IndentLevelScope())
{
DrawTypeProperties();
}
}
private void DrawTypeProperties()
{
if (target is RoughPositionConstraint)
{
DrawProperties("constrainedAxis", "positionOffset", "offsetInSourceSpace");
}
else if (target is RoughRotationConstraint)
{
DrawProperties("constrainedAxis", "rotationOffset");
}
else if (target is RoughScaleConstraint)
{
DrawProperties("constrainedAxis", "scaleOffset");
}
else if (target is RoughParentConstraint)
{
DrawProperties("constrainedPositionAxis", "positionOffset", "positionOffsetInSourceSpace", "constrainedRotationAxis", "rotationOffset");
}
else if (target is RoughAimConstraint)
{
DrawProperties("aimAxis", "upAxis", "worldUpVector", "rotationOffset");
}
else if (target is RoughLookAtConstraint)
{
DrawProperties("forwardAxis", "worldUpVector", "targetPositionOffset", "rotationOffset");
}
}
private void DrawProperties(params string[] propertyNames)
{
for (var i = 0; i < propertyNames.Length; i++)
{
var property = serializedObject.FindProperty(propertyNames[i]);
if (property != null)
{
EditorGUILayout.PropertyField(property, new GUIContent(ObjectNames.NicifyVariableName(property.name)));
}
}
}
private void ApplyAction(string undoName, System.Action<RoughConstraintBase> action)
{
serializedObject.ApplyModifiedProperties();
foreach (var selectedTarget in targets)
{
var constraint = (RoughConstraintBase)selectedTarget;
Undo.RecordObject(constraint.transform, undoName);
action(constraint);
EditorUtility.SetDirty(constraint.transform);
}
}
private void ActivateConstraint(RoughConstraintBase constraint)
{
Undo.RecordObject(constraint, "Activate Rough Constraint");
constraint.ActivateConstraint();
EditorUtility.SetDirty(constraint);
}
private void ZeroOffsets()
{
serializedObject.ApplyModifiedProperties();
Undo.RecordObjects(targets, "Zero Rough Constraint Offsets");
SetVector3IfExists("positionOffset", Vector3.zero);
SetVector3IfExists("rotationOffset", Vector3.zero);
SetVector3IfExists("scaleOffset", Vector3.zero);
SetVector3IfExists("targetPositionOffset", Vector3.zero);
serializedObject.ApplyModifiedProperties();
}
private void SetVector3IfExists(string propertyName, Vector3 value)
{
var property = serializedObject.FindProperty(propertyName);
if (property != null)
{
property.vector3Value = value;
}
}
private void DrawSourceHeader(Rect rect)
{
var objectRect = new Rect(rect.x + 14f, rect.y, rect.width - 88f, rect.height);
var weightRect = new Rect(rect.xMax - 70f, rect.y, 68f, rect.height);
EditorGUI.LabelField(objectRect, "Sources");
EditorGUI.LabelField(weightRect, "Weight");
}
private void DrawSourceElement(Rect rect, int index, bool isActive, bool isFocused)
{
var element = sources.GetArrayElementAtIndex(index);
var sourceTransform = element.FindPropertyRelative("sourceTransform");
var sourceWeight = element.FindPropertyRelative("weight");
rect.y += 3f;
rect.height = EditorGUIUtility.singleLineHeight;
if (sourceTransform == null || sourceWeight == null)
{
EditorGUI.PropertyField(rect, element, GUIContent.none);
return;
}
var weightWidth = 70f;
var gap = 6f;
var targetRect = new Rect(rect.x, rect.y, rect.width - weightWidth - gap, rect.height);
var weightRect = new Rect(targetRect.xMax + gap, rect.y, weightWidth, rect.height);
EditorGUI.PropertyField(targetRect, sourceTransform, GUIContent.none);
sourceWeight.floatValue = Mathf.Clamp01(EditorGUI.FloatField(weightRect, sourceWeight.floatValue));
}
private void OnAddSource(ReorderableList list)
{
sources.arraySize++;
var element = sources.GetArrayElementAtIndex(sources.arraySize - 1);
var sourceTransform = element.FindPropertyRelative("sourceTransform");
var sourceWeight = element.FindPropertyRelative("weight");
if (sourceTransform != null)
{
sourceTransform.objectReferenceValue = null;
}
if (sourceWeight != null)
{
sourceWeight.floatValue = 1f;
}
}
private string GetSettingsTitle()
{
if (target is RoughLookAtConstraint)
{
return "Look At";
}
var name = target.GetType().Name;
name = name.Replace("Rough", string.Empty).Replace("Constraint", string.Empty);
return ObjectNames.NicifyVariableName(name);
}
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: b69eed063de14090a1e439e86149a6a6

View File

@ -0,0 +1,44 @@
using Streamingle.Utils.RoughConstraints;
using UnityEditor;
using UnityEngine;
namespace Streamingle.Utils.RoughConstraintsEditor
{
[CustomPropertyDrawer(typeof(Vector3Bool))]
public sealed class Vector3BoolDrawer : UnityEditor.PropertyDrawer
{
public override void OnGUI(Rect position, SerializedProperty property, GUIContent label)
{
var x = property.FindPropertyRelative("x");
var y = property.FindPropertyRelative("y");
var z = property.FindPropertyRelative("z");
position = EditorGUI.PrefixLabel(position, label);
var previousIndent = EditorGUI.indentLevel;
EditorGUI.indentLevel = 0;
var gap = 4f;
var width = (position.width - gap * 2f) / 3f;
DrawToggle(new Rect(position.x, position.y, width, position.height), x, "X");
DrawToggle(new Rect(position.x + width + gap, position.y, width, position.height), y, "Y");
DrawToggle(new Rect(position.x + (width + gap) * 2f, position.y, width, position.height), z, "Z");
EditorGUI.indentLevel = previousIndent;
}
private static void DrawToggle(Rect rect, SerializedProperty property, string label)
{
var enabled = property.boolValue;
var previousColor = GUI.backgroundColor;
GUI.backgroundColor = enabled ? new Color(0.55f, 0.82f, 1f) : previousColor;
if (GUI.Button(rect, label, EditorStyles.miniButton))
{
property.boolValue = !enabled;
}
GUI.backgroundColor = previousColor;
}
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: 78d95149f81448ad809828060b5800ce

8
Editor/Gaze.meta Normal file
View File

@ -0,0 +1,8 @@
fileFormatVersion: 2
guid: ecc5b45bd3ebb3548a0d9c27af991f81
folderAsset: yes
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

View File

@ -0,0 +1,3 @@
using System.Runtime.CompilerServices;
[assembly: InternalsVisibleTo("Streamingle.Gaze.Tests.Editor")]

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: ff643376513f90b4f88310d09238aa75

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: faf76a1077abde041aee942d9ec420ee

View File

@ -0,0 +1,473 @@
using System;
using System.Collections.Generic;
using UnityEditor;
using UnityEditor.SceneManagement;
using UnityEngine;
using UnityEngine.SceneManagement;
namespace Streamingle.Gaze.Editor
{
/// <summary>
/// Owns the temporary blendshape writes used by the calibration UI.
/// The session never modifies the profile or scene and always attempts to
/// restore the weights that were present when it started.
/// </summary>
internal sealed class GazeCalibrationPreviewSession : IDisposable
{
private int[] blendShapeIndices = Array.Empty<int>();
private string[] resolvedBlendShapeNames = Array.Empty<string>();
private float[] originalWeights = Array.Empty<float>();
private float[] previewWeights = Array.Empty<float>();
private Mesh sourceMesh;
private bool hasPreview;
private bool eventsSubscribed;
private bool driverOverrideAcquired;
private BlendshapeGazeProfile driverProfile;
internal event Action Stopped;
public bool IsActive { get; private set; }
public BlendshapeGazeDriver Driver { get; private set; }
public SkinnedMeshRenderer Renderer { get; private set; }
public BlendshapeGazeProfile Profile { get; private set; }
public Vector2 PreviewCoordinates { get; private set; }
public bool IsFreePreview { get; private set; }
public bool Start(BlendshapeGazeDriver driver, out string error)
{
return Start(driver, null, out error);
}
internal bool Start(
BlendshapeGazeDriver driver,
BlendshapeGazeProfile previewProfile,
out string error)
{
Stop();
error = null;
if (Application.isPlaying || EditorApplication.isPlayingOrWillChangePlaymode)
{
error = "Calibration preview is only available in Edit Mode.";
return false;
}
if (driver == null)
{
error = "A Blendshape Gaze Driver is required.";
return false;
}
var renderer = driver.TargetRenderer;
if (renderer == null)
{
error = "The driver does not have a Target Renderer.";
return false;
}
var mesh = renderer.sharedMesh;
if (mesh == null)
{
error = "The Target Renderer does not have a shared mesh.";
return false;
}
var assignedProfile = driver.Profile;
if (assignedProfile == null)
{
error = "The driver does not have a Gaze Profile.";
return false;
}
var profile = previewProfile != null ? previewProfile : assignedProfile;
if (!profile.IsCompatibleWith(mesh))
{
error = "The Gaze Profile was calibrated for a different source mesh.";
return false;
}
var channels = profile.Channels;
if (channels == null || channels.Count == 0)
{
error = "The Gaze Profile does not contain any blendshape channels.";
return false;
}
var indices = new int[channels.Count];
var names = new string[channels.Count];
var originals = new float[channels.Count];
for (var channelIndex = 0; channelIndex < channels.Count; channelIndex++)
{
var channel = channels[channelIndex];
if (channel == null || string.IsNullOrWhiteSpace(channel.BlendShapeName))
{
error = $"Gaze channel {channelIndex + 1} does not have a blendshape name.";
return false;
}
var blendShapeIndex = profile.FindBlendShapeIndex(mesh, channelIndex);
if (blendShapeIndex < 0 || blendShapeIndex >= mesh.blendShapeCount)
{
error = $"Blendshape '{channel.BlendShapeName}' was not found on '{mesh.name}'.";
return false;
}
indices[channelIndex] = blendShapeIndex;
names[channelIndex] = mesh.GetBlendShapeName(blendShapeIndex);
}
// The driver relinquishes its current gaze result only after the entire
// profile-to-mesh mapping has passed validation. Capture the baseline
// after that release so Stop restores the actual external facial pose.
driver.BeginCalibrationPreviewOverride();
try
{
for (var channelIndex = 0; channelIndex < channels.Count; channelIndex++)
{
var blendShapeIndex = indices[channelIndex];
originals[channelIndex] = renderer.GetBlendShapeWeight(blendShapeIndex);
}
Driver = driver;
Renderer = renderer;
Profile = profile;
driverProfile = assignedProfile;
sourceMesh = mesh;
blendShapeIndices = indices;
resolvedBlendShapeNames = names;
originalWeights = originals;
previewWeights = new float[channels.Count];
PreviewCoordinates = Vector2.zero;
IsFreePreview = false;
hasPreview = false;
driverOverrideAcquired = true;
IsActive = true;
SubscribeEvents();
return true;
}
catch (Exception exception)
{
driver.EndCalibrationPreviewOverride();
error = $"Could not start calibration preview: {exception.Message}";
return false;
}
}
public void PreviewSample(GazeCalibrationPoint point)
{
if (!IsActive || !IsSessionValid())
return;
var sampleIndex = (int)point;
var samples = Profile.Samples;
if (sampleIndex < 0 || sampleIndex >= samples.Count)
return;
var sample = samples[sampleIndex];
if (sample == null || !sample.Captured || sample.Weights.Count != previewWeights.Length)
return;
PreviewWeights(sample.Weights, point);
}
public void PreviewWeights(IReadOnlyList<float> weights, GazeCalibrationPoint point)
{
if (!IsActive || !IsSessionValid() || !IsValidPoint(point)
|| !TrySetPreviewWeights(weights, point == GazeCalibrationPoint.Center))
return;
PreviewCoordinates = CoordinatesFor(point);
IsFreePreview = false;
hasPreview = true;
ApplyPreviewWeights();
}
public void PreviewNormalized(Vector2 normalized)
{
if (!IsActive || !IsSessionValid())
return;
normalized = new Vector2(
Mathf.Clamp(normalized.x, -1f, 1f),
Mathf.Clamp(normalized.y, -1f, 1f));
var evaluated = new float[previewWeights.Length];
var direction = new GazeDirectionResult(
Vector3.forward,
Vector3.forward,
0f,
0f,
0f,
0f,
normalized.x,
normalized.y);
// TryEvaluate also guarantees that all nine calibration samples exist.
// Correctives are intentionally included so this is an exact preview of
// the completed profile rather than only its eye-look subset.
if (!Profile.TryEvaluate(direction, true, evaluated))
return;
if (!TrySetPreviewWeights(evaluated, false))
return;
PreviewCoordinates = normalized;
IsFreePreview = true;
hasPreview = true;
ApplyPreviewWeights();
}
public float[] GetOriginalWeightsCopy()
{
return (float[])originalWeights.Clone();
}
public void Stop()
{
var wasActive = IsActive;
UnsubscribeEvents();
var overriddenDriver = Driver;
try
{
if (IsActive)
RestoreOriginalWeights();
}
finally
{
// Restoring while the override is still held guarantees that a
// Timeline/editor evaluation cannot race the final face reset.
if (driverOverrideAcquired && overriddenDriver != null)
overriddenDriver.EndCalibrationPreviewOverride();
IsActive = false;
Driver = null;
Renderer = null;
Profile = null;
driverProfile = null;
sourceMesh = null;
blendShapeIndices = Array.Empty<int>();
resolvedBlendShapeNames = Array.Empty<string>();
originalWeights = Array.Empty<float>();
previewWeights = Array.Empty<float>();
PreviewCoordinates = Vector2.zero;
IsFreePreview = false;
hasPreview = false;
driverOverrideAcquired = false;
if (wasActive)
Stopped?.Invoke();
}
}
public void Dispose()
{
Stop();
}
private void OnEditorUpdate()
{
if (!IsActive)
return;
if (EditorApplication.isPlayingOrWillChangePlaymode || !IsSessionValid())
{
Stop();
return;
}
if (hasPreview)
ApplyPreviewWeights();
}
private void OnBeforeAssemblyReload()
{
Stop();
}
private void OnEditorQuitting()
{
Stop();
}
private void OnPlayModeStateChanged(PlayModeStateChange state)
{
Stop();
}
private void OnSceneSaving(Scene scene, string path)
{
Stop();
}
private void OnPrefabSaving(GameObject root)
{
Stop();
}
private void SubscribeEvents()
{
if (eventsSubscribed)
return;
EditorApplication.update += OnEditorUpdate;
EditorApplication.quitting += OnEditorQuitting;
EditorApplication.playModeStateChanged += OnPlayModeStateChanged;
AssemblyReloadEvents.beforeAssemblyReload += OnBeforeAssemblyReload;
EditorSceneManager.sceneSaving += OnSceneSaving;
PrefabStage.prefabSaving += OnPrefabSaving;
eventsSubscribed = true;
}
private void UnsubscribeEvents()
{
if (!eventsSubscribed)
return;
EditorApplication.update -= OnEditorUpdate;
EditorApplication.quitting -= OnEditorQuitting;
EditorApplication.playModeStateChanged -= OnPlayModeStateChanged;
AssemblyReloadEvents.beforeAssemblyReload -= OnBeforeAssemblyReload;
EditorSceneManager.sceneSaving -= OnSceneSaving;
PrefabStage.prefabSaving -= OnPrefabSaving;
eventsSubscribed = false;
}
private bool IsSessionValid()
{
if (!IsActive || Driver == null || Renderer == null || Profile == null
|| driverProfile == null || sourceMesh == null)
return false;
// Cached indices belong exclusively to this exact mesh. Never use them
// after a renderer mesh swap, even if the replacement has matching names.
if (Renderer.sharedMesh != sourceMesh
|| Driver.TargetRenderer != Renderer
|| Driver.Profile != driverProfile
|| !Profile.IsCompatibleWith(sourceMesh))
return false;
var channels = Profile.Channels;
if (channels == null || channels.Count != blendShapeIndices.Length
|| channels.Count != resolvedBlendShapeNames.Length)
return false;
for (var channelIndex = 0; channelIndex < channels.Count; channelIndex++)
{
var channel = channels[channelIndex];
var blendShapeIndex = blendShapeIndices[channelIndex];
if (channel == null || blendShapeIndex < 0 || blendShapeIndex >= sourceMesh.blendShapeCount
|| !string.Equals(
sourceMesh.GetBlendShapeName(blendShapeIndex),
resolvedBlendShapeNames[channelIndex],
StringComparison.Ordinal)
|| Profile.FindBlendShapeIndex(sourceMesh, channelIndex) != blendShapeIndices[channelIndex])
return false;
}
return previewWeights.Length == blendShapeIndices.Length
&& originalWeights.Length == blendShapeIndices.Length;
}
private void ApplyPreviewWeights()
{
if (!hasPreview || !IsSessionValid())
return;
for (var channelIndex = 0; channelIndex < blendShapeIndices.Length; channelIndex++)
Renderer.SetBlendShapeWeight(blendShapeIndices[channelIndex], previewWeights[channelIndex]);
}
private void RestoreOriginalWeights()
{
if (Renderer == null || sourceMesh == null || Renderer.sharedMesh != sourceMesh)
return;
var count = Mathf.Min(blendShapeIndices.Length, originalWeights.Length);
if (count != resolvedBlendShapeNames.Length)
return;
// Validate the complete cached index/name mapping before writing any
// value. A mesh rebuilt in place may reuse an index for another shape.
for (var channelIndex = 0; channelIndex < count; channelIndex++)
{
var blendShapeIndex = blendShapeIndices[channelIndex];
if (blendShapeIndex < 0 || blendShapeIndex >= sourceMesh.blendShapeCount
|| !string.Equals(
sourceMesh.GetBlendShapeName(blendShapeIndex),
resolvedBlendShapeNames[channelIndex],
StringComparison.Ordinal))
return;
}
for (var channelIndex = 0; channelIndex < count; channelIndex++)
Renderer.SetBlendShapeWeight(blendShapeIndices[channelIndex], originalWeights[channelIndex]);
}
private bool TrySetPreviewWeights(IReadOnlyList<float> weights, bool inputRepresentsCenter)
{
var channels = Profile != null ? Profile.Channels : null;
if (weights == null || channels == null || weights.Count != previewWeights.Length
|| channels.Count != previewWeights.Length || originalWeights.Length != previewWeights.Length)
return false;
var translated = new float[weights.Count];
for (var channelIndex = 0; channelIndex < weights.Count; channelIndex++)
{
var sampleWeight = weights[channelIndex];
if (!IsFinite(sampleWeight))
return false;
var channel = channels[channelIndex];
if (channel == null)
return false;
if (channel.Usage == GazeChannelUsage.Corrective
&& channel.CorrectiveBlendMode == GazeCorrectiveBlendMode.AdditiveFromCenter)
{
var centerWeight = inputRepresentsCenter
? sampleWeight
: Profile.IsSampleCaptured(GazeCalibrationPoint.Center)
? Profile.GetSampleWeight(GazeCalibrationPoint.Center, channelIndex)
: originalWeights[channelIndex];
if (!IsFinite(centerWeight))
return false;
translated[channelIndex] = originalWeights[channelIndex] + sampleWeight - centerWeight;
}
else
{
translated[channelIndex] = sampleWeight;
}
if (!IsFinite(translated[channelIndex]))
return false;
}
Array.Copy(translated, previewWeights, translated.Length);
return true;
}
private static Vector2 CoordinatesFor(GazeCalibrationPoint point)
{
var index = (int)point;
return new Vector2(index % 3 - 1, index / 3 - 1);
}
private static bool IsValidPoint(GazeCalibrationPoint point)
{
var index = (int)point;
return index >= 0 && index < 9;
}
private static bool IsFinite(float value)
{
return !float.IsNaN(value) && !float.IsInfinity(value);
}
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: 4fd84d575a717f54ba289016048bb22b

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: 6b4ab3e2eac3c4e46868fdc5ee5a449e

View File

@ -0,0 +1,426 @@
using System;
using System.Collections.Generic;
using UnityEngine;
namespace Streamingle.Gaze.Editor
{
/// <summary>
/// Result of an editor-only gaze-origin estimation pass.
/// </summary>
internal readonly struct GazeOriginEstimate
{
internal GazeOriginEstimate(
Vector3 worldPosition,
Vector3 leftEyeWorldPosition,
Vector3 rightEyeWorldPosition,
bool usedEyeBlendShapes,
int leftVertexCount,
int rightVertexCount,
string message)
{
WorldPosition = worldPosition;
LeftEyeWorldPosition = leftEyeWorldPosition;
RightEyeWorldPosition = rightEyeWorldPosition;
UsedEyeBlendShapes = usedEyeBlendShapes;
LeftVertexCount = leftVertexCount;
RightVertexCount = rightVertexCount;
Message = message;
}
internal Vector3 WorldPosition { get; }
internal Vector3 LeftEyeWorldPosition { get; }
internal Vector3 RightEyeWorldPosition { get; }
internal bool UsedEyeBlendShapes { get; }
internal int LeftVertexCount { get; }
internal int RightVertexCount { get; }
internal string Message { get; }
}
/// <summary>
/// Estimates the midpoint between the eyes from the vertices influenced by the configured
/// ARKit eyeLook blend shapes. This is deliberately editor-only: BakeMesh is never used by
/// the runtime gaze driver.
/// </summary>
internal static class GazeOriginEstimator
{
private const float MinimumDelta = 1e-7f;
private const int MinimumVerticesPerEye = 3;
internal static GazeOriginEstimate Estimate(
SkinnedMeshRenderer renderer,
IReadOnlyList<string> configuredChannelNames)
{
if (renderer == null)
{
return new GazeOriginEstimate(
Vector3.zero,
Vector3.zero,
Vector3.zero,
false,
0,
0,
"Target Renderer is not assigned.");
}
Mesh sourceMesh = renderer.sharedMesh;
if (sourceMesh == null || sourceMesh.vertexCount == 0)
{
return Fallback(renderer, "The renderer has no readable shared mesh.");
}
var leftShapeIndices = new HashSet<int>();
var rightShapeIndices = new HashSet<int>();
CollectShapeIndices(sourceMesh, configuredChannelNames, leftShapeIndices, rightShapeIndices);
if (leftShapeIndices.Count == 0 || rightShapeIndices.Count == 0)
{
return Fallback(
renderer,
"Could not find configured eyeLook blend shapes for both eyes. Used Renderer bounds center.");
}
int vertexCount = sourceMesh.vertexCount;
var leftMaximumDeltas = new float[vertexCount];
var rightMaximumDeltas = new float[vertexCount];
var deltaVertices = new Vector3[vertexCount];
var deltaNormals = new Vector3[vertexCount];
var deltaTangents = new Vector3[vertexCount];
AccumulateMaximumDeltas(
sourceMesh,
leftShapeIndices,
leftMaximumDeltas,
deltaVertices,
deltaNormals,
deltaTangents);
AccumulateMaximumDeltas(
sourceMesh,
rightShapeIndices,
rightMaximumDeltas,
deltaVertices,
deltaNormals,
deltaTangents);
Mesh bakedMesh = null;
try
{
bakedMesh = new Mesh { name = "GazeOriginEstimator_TemporaryMesh" };
renderer.BakeMesh(bakedMesh);
Vector3[] neutralVertices = bakedMesh.vertices;
if (neutralVertices == null || neutralVertices.Length != vertexCount)
{
return Fallback(renderer, "BakeMesh vertex count did not match the source mesh.");
}
bool hasLeft = TryCalculateWeightedCentroid(
renderer.transform,
neutralVertices,
leftMaximumDeltas,
out Vector3 leftCenter,
out int leftVertexCount);
bool hasRight = TryCalculateWeightedCentroid(
renderer.transform,
neutralVertices,
rightMaximumDeltas,
out Vector3 rightCenter,
out int rightVertexCount);
if (!hasLeft || !hasRight)
{
return Fallback(
renderer,
"Eye blend shapes did not contain enough significant vertices. Used Renderer bounds center.",
leftVertexCount,
rightVertexCount);
}
// Deliberately give both eyes equal influence. A larger or stronger blend shape on one
// side must not pull the gaze origin away from the center of the face.
Vector3 midpoint = (leftCenter + rightCenter) * 0.5f;
return new GazeOriginEstimate(
midpoint,
leftCenter,
rightCenter,
true,
leftVertexCount,
rightVertexCount,
$"Estimated from {leftVertexCount} left-eye and {rightVertexCount} right-eye vertices.");
}
catch (Exception exception)
{
return Fallback(renderer, $"Eye vertex estimation failed: {exception.Message}");
}
finally
{
if (bakedMesh != null)
{
UnityEngine.Object.DestroyImmediate(bakedMesh);
}
}
}
internal static int FindBlendShapeIndex(Mesh mesh, string configuredName)
{
if (mesh == null || string.IsNullOrWhiteSpace(configuredName))
{
return -1;
}
string normalizedConfiguredName = Normalize(configuredName);
int suffixMatch = -1;
for (int index = 0; index < mesh.blendShapeCount; index++)
{
string meshName = mesh.GetBlendShapeName(index);
if (string.Equals(meshName, configuredName, StringComparison.OrdinalIgnoreCase))
{
return index;
}
if (meshName.EndsWith(configuredName, StringComparison.OrdinalIgnoreCase))
{
suffixMatch = index;
continue;
}
string normalizedMeshName = Normalize(meshName);
if (normalizedMeshName == normalizedConfiguredName)
{
return index;
}
if (normalizedMeshName.EndsWith(normalizedConfiguredName, StringComparison.Ordinal))
{
suffixMatch = index;
}
}
return suffixMatch;
}
private static void CollectShapeIndices(
Mesh mesh,
IReadOnlyList<string> configuredChannelNames,
ISet<int> leftShapeIndices,
ISet<int> rightShapeIndices)
{
if (configuredChannelNames != null)
{
for (int index = 0; index < configuredChannelNames.Count; index++)
{
string channelName = configuredChannelNames[index];
int blendShapeIndex = FindBlendShapeIndex(mesh, channelName);
if (blendShapeIndex < 0)
{
continue;
}
string sideName = string.IsNullOrWhiteSpace(channelName)
? mesh.GetBlendShapeName(blendShapeIndex)
: channelName;
if (IsLeftChannel(sideName))
{
leftShapeIndices.Add(blendShapeIndex);
}
else if (IsRightChannel(sideName))
{
rightShapeIndices.Add(blendShapeIndex);
}
}
}
// A newly created profile has no bindings yet. Scanning the mesh for the standard suffixes
// still gives Auto Create a useful result before the user presses Sync Channels.
if (leftShapeIndices.Count > 0 && rightShapeIndices.Count > 0)
{
return;
}
for (int index = 0; index < mesh.blendShapeCount; index++)
{
string name = mesh.GetBlendShapeName(index);
string normalizedName = Normalize(name);
if (!normalizedName.Contains("eyelook"))
{
continue;
}
if (IsLeftChannel(name))
{
leftShapeIndices.Add(index);
}
else if (IsRightChannel(name))
{
rightShapeIndices.Add(index);
}
}
}
private static void AccumulateMaximumDeltas(
Mesh mesh,
IEnumerable<int> shapeIndices,
float[] maximumDeltas,
Vector3[] deltaVertices,
Vector3[] deltaNormals,
Vector3[] deltaTangents)
{
foreach (int shapeIndex in shapeIndices)
{
int frameCount = mesh.GetBlendShapeFrameCount(shapeIndex);
for (int frameIndex = 0; frameIndex < frameCount; frameIndex++)
{
mesh.GetBlendShapeFrameVertices(
shapeIndex,
frameIndex,
deltaVertices,
deltaNormals,
deltaTangents);
for (int vertexIndex = 0; vertexIndex < deltaVertices.Length; vertexIndex++)
{
float magnitude = deltaVertices[vertexIndex].magnitude;
if (magnitude > maximumDeltas[vertexIndex])
{
maximumDeltas[vertexIndex] = magnitude;
}
}
}
}
}
private static bool TryCalculateWeightedCentroid(
Transform rendererTransform,
IReadOnlyList<Vector3> neutralVertices,
IReadOnlyList<float> maximumDeltas,
out Vector3 worldCentroid,
out int includedVertexCount)
{
var positiveDeltas = new List<float>();
for (int index = 0; index < maximumDeltas.Count; index++)
{
float delta = maximumDeltas[index];
if (delta > MinimumDelta && IsFinite(delta))
{
positiveDeltas.Add(delta);
}
}
if (positiveDeltas.Count < MinimumVerticesPerEye)
{
worldCentroid = Vector3.zero;
includedVertexCount = 0;
return false;
}
positiveDeltas.Sort();
float median = Percentile(positiveDeltas, 0.5f);
float percentile95 = Percentile(positiveDeltas, 0.95f);
float percentile99 = Percentile(positiveDeltas, 0.99f);
// Remove tiny cross-face noise and reject pathological spikes. Valid high-motion eye
// vertices are retained and winsorized so one vertex cannot dominate the centroid.
float lowerThreshold = Mathf.Max(MinimumDelta, Mathf.Max(median * 0.05f, percentile95 * 0.015f));
float upperThreshold = Mathf.Max(percentile99 * 4f, lowerThreshold);
float maximumWeight = Mathf.Max(percentile95 * 2f, lowerThreshold);
Vector3 weightedPosition = Vector3.zero;
float weightSum = 0f;
includedVertexCount = 0;
for (int index = 0; index < maximumDeltas.Count; index++)
{
float delta = maximumDeltas[index];
if (!IsFinite(delta) || delta < lowerThreshold || delta > upperThreshold)
{
continue;
}
float weight = Mathf.Min(delta, maximumWeight);
weightedPosition += rendererTransform.TransformPoint(neutralVertices[index]) * weight;
weightSum += weight;
includedVertexCount++;
}
if (includedVertexCount < MinimumVerticesPerEye || weightSum <= Mathf.Epsilon)
{
worldCentroid = Vector3.zero;
return false;
}
worldCentroid = weightedPosition / weightSum;
return IsFinite(worldCentroid);
}
private static float Percentile(IReadOnlyList<float> sortedValues, float percentile)
{
if (sortedValues.Count == 0)
{
return 0f;
}
float position = Mathf.Clamp01(percentile) * (sortedValues.Count - 1);
int lowerIndex = Mathf.FloorToInt(position);
int upperIndex = Mathf.CeilToInt(position);
return Mathf.Lerp(sortedValues[lowerIndex], sortedValues[upperIndex], position - lowerIndex);
}
private static GazeOriginEstimate Fallback(
SkinnedMeshRenderer renderer,
string message,
int leftVertexCount = 0,
int rightVertexCount = 0)
{
Vector3 center = renderer != null ? renderer.bounds.center : Vector3.zero;
return new GazeOriginEstimate(
center,
center,
center,
false,
leftVertexCount,
rightVertexCount,
message);
}
private static bool IsLeftChannel(string name)
{
return Normalize(name).EndsWith("left", StringComparison.Ordinal);
}
private static bool IsRightChannel(string name)
{
return Normalize(name).EndsWith("right", StringComparison.Ordinal);
}
private static string Normalize(string value)
{
if (string.IsNullOrEmpty(value))
{
return string.Empty;
}
var characters = new char[value.Length];
int count = 0;
for (int index = 0; index < value.Length; index++)
{
char character = value[index];
if (!char.IsLetterOrDigit(character))
{
continue;
}
characters[count++] = char.ToLowerInvariant(character);
}
return new string(characters, 0, count);
}
private static bool IsFinite(Vector3 value)
{
return IsFinite(value.x) && IsFinite(value.y) && IsFinite(value.z);
}
private static bool IsFinite(float value)
{
return !float.IsNaN(value) && !float.IsInfinity(value);
}
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: 4e767c5be201bbc4ca28ed9149cd78a5

View File

@ -0,0 +1,18 @@
{
"name": "Streamingle.Gaze.Editor",
"rootNamespace": "Streamingle.Gaze.Editor",
"references": [
"Streamingle.Gaze.Runtime"
],
"includePlatforms": [
"Editor"
],
"excludePlatforms": [],
"allowUnsafeCode": false,
"overrideReferences": false,
"precompiledReferences": [],
"autoReferenced": true,
"defineConstraints": [],
"versionDefines": [],
"noEngineReferences": false
}

View File

@ -0,0 +1,7 @@
fileFormatVersion: 2
guid: 161a6877619d01f4fbf89fc2896a2b19
AssemblyDefinitionImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

View File

@ -0,0 +1,19 @@
{
"name": "Streamingle.Utilities.Editor",
"rootNamespace": "Streamingle.Utilities.Editor",
"references": [
"Streamingle.Utilities.Runtime",
"Streamingle.Subtitles.Runtime"
],
"includePlatforms": [
"Editor"
],
"excludePlatforms": [],
"allowUnsafeCode": false,
"overrideReferences": false,
"precompiledReferences": [],
"autoReferenced": true,
"defineConstraints": [],
"versionDefines": [],
"noEngineReferences": false
}

View File

@ -0,0 +1,7 @@
fileFormatVersion: 2
guid: f2adcdd669f914b439f27fb8dd249d99
AssemblyDefinitionImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

View File

@ -0,0 +1,188 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text.RegularExpressions;
using TMPro;
using UnityEditor;
using UnityEditor.SceneManagement;
using UnityEngine;
using UnityEngine.Playables;
using UnityEngine.Timeline;
namespace Streamingle.Subtitles.Editor
{
public static class SrtSubtitleInstaller
{
public const string GeneratedTrackName = "Subtitles [Generated]";
private static readonly Regex TimestampPattern = new Regex(
@"^(?<sh>\d+):(?<sm>\d{2}):(?<ss>\d{2}),(?<sms>\d{3})\s*-->\s*(?<eh>\d+):(?<em>\d{2}):(?<es>\d{2}),(?<ems>\d{3})$",
RegexOptions.Compiled | RegexOptions.CultureInvariant);
public static void Install(
PlayableDirector director,
TMP_Text target,
string srtAssetPath,
double offsetSeconds)
{
if (director == null)
throw new ArgumentNullException(nameof(director));
if (target == null)
throw new ArgumentNullException(nameof(target));
if (director.playableAsset is not TimelineAsset timeline)
throw new InvalidOperationException("The PlayableDirector must reference a TimelineAsset.");
if (double.IsNaN(offsetSeconds) || double.IsInfinity(offsetSeconds))
throw new ArgumentOutOfRangeException(nameof(offsetSeconds));
var fullSrtPath = ResolveProjectPath(srtAssetPath);
if (!File.Exists(fullSrtPath))
throw new FileNotFoundException("SRT file was not found.", fullSrtPath);
var cues = Parse(File.ReadAllText(fullSrtPath));
if (cues.Count == 0)
throw new InvalidDataException("The SRT file contains no subtitle cues.");
var undoGroup = Undo.GetCurrentGroup();
Undo.SetCurrentGroupName("Install generated subtitle track");
try
{
Undo.RegisterCompleteObjectUndo(timeline, "Replace generated subtitle track");
Undo.RecordObject(director, "Bind generated subtitle track");
RemoveGeneratedTracks(timeline, director);
var track = timeline.CreateTrack<SubtitleTrack>(null, GeneratedTrackName);
foreach (var cue in cues)
{
var start = Math.Max(0d, cue.StartSeconds + offsetSeconds);
var clip = track.CreateClip<SubtitleClip>();
clip.start = start;
clip.duration = cue.EndSeconds - cue.StartSeconds;
clip.displayName = CreateDisplayName(cue.Text);
if (clip.asset is SubtitleClip subtitleAsset)
{
subtitleAsset.Text = cue.Text;
EditorUtility.SetDirty(subtitleAsset);
}
}
director.SetGenericBinding(track, target);
EditorUtility.SetDirty(track);
EditorUtility.SetDirty(timeline);
EditorUtility.SetDirty(director);
EditorSceneManager.MarkSceneDirty(director.gameObject.scene);
AssetDatabase.SaveAssetIfDirty(timeline);
director.RebuildGraph();
director.Evaluate();
SceneView.RepaintAll();
}
finally
{
Undo.CollapseUndoOperations(undoGroup);
}
}
public static IReadOnlyList<SrtCue> Parse(string srtContents)
{
if (string.IsNullOrWhiteSpace(srtContents))
return Array.Empty<SrtCue>();
var normalized = srtContents.Replace("\r\n", "\n").Replace('\r', '\n').Trim();
var blocks = Regex.Split(normalized, @"\n{2,}");
var cues = new List<SrtCue>(blocks.Length);
for (var blockIndex = 0; blockIndex < blocks.Length; blockIndex++)
{
var lines = blocks[blockIndex]
.Split('\n')
.Select(line => line.TrimEnd())
.ToArray();
var timestampLineIndex = Array.FindIndex(lines, line => line.Contains("-->"));
if (timestampLineIndex < 0 || timestampLineIndex + 1 >= lines.Length)
throw new FormatException($"Invalid SRT cue at block {blockIndex + 1}.");
var match = TimestampPattern.Match(lines[timestampLineIndex].Trim());
if (!match.Success)
throw new FormatException($"Invalid SRT timestamp at block {blockIndex + 1}.");
var start = ParseTimestamp(match, "s");
var end = ParseTimestamp(match, "e");
if (end <= start)
throw new FormatException($"SRT cue {blockIndex + 1} must end after it starts.");
var text = string.Join("\n", lines.Skip(timestampLineIndex + 1)).Trim();
if (text.Length == 0)
continue;
if (cues.Count > 0 && start < cues[cues.Count - 1].EndSeconds)
throw new FormatException($"SRT cue {blockIndex + 1} overlaps the previous cue.");
cues.Add(new SrtCue(start, end, text));
}
return cues;
}
private static void RemoveGeneratedTracks(TimelineAsset timeline, PlayableDirector director)
{
var generatedTracks = timeline.GetRootTracks()
.OfType<SubtitleTrack>()
.Where(track => track.name == GeneratedTrackName)
.ToArray();
foreach (var track in generatedTracks)
{
director.ClearGenericBinding(track);
timeline.DeleteTrack(track);
}
}
private static string ResolveProjectPath(string path)
{
if (string.IsNullOrWhiteSpace(path))
throw new ArgumentException("An SRT asset path is required.", nameof(path));
if (Path.IsPathRooted(path))
return Path.GetFullPath(path);
var projectRoot = Directory.GetParent(Application.dataPath)?.FullName
?? throw new InvalidOperationException("Could not resolve the Unity project root.");
return Path.GetFullPath(Path.Combine(projectRoot, path));
}
private static double ParseTimestamp(Match match, string prefix)
{
var hours = int.Parse(match.Groups[prefix + "h"].Value);
var minutes = int.Parse(match.Groups[prefix + "m"].Value);
var seconds = int.Parse(match.Groups[prefix + "s"].Value);
var milliseconds = int.Parse(match.Groups[prefix + "ms"].Value);
return hours * 3600d + minutes * 60d + seconds + milliseconds / 1000d;
}
private static string CreateDisplayName(string text)
{
var singleLine = Regex.Replace(text ?? string.Empty, @"\s+", " ").Trim();
return singleLine.Length <= 36 ? singleLine : singleLine.Substring(0, 35) + "…";
}
public readonly struct SrtCue
{
public SrtCue(double startSeconds, double endSeconds, string text)
{
StartSeconds = startSeconds;
EndSeconds = endSeconds;
Text = text;
}
public double StartSeconds { get; }
public double EndSeconds { get; }
public string Text { get; }
}
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: cdb14b2eda3be7741a811ba8be8ab1b0

3
LICENSE.md Normal file
View File

@ -0,0 +1,3 @@
# License
License terms will be added before the first external release.

7
LICENSE.md.meta Normal file
View File

@ -0,0 +1,7 @@
fileFormatVersion: 2
guid: 971659ddca6a5d644821fe305d634f23
TextScriptImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

27
README.md Normal file
View File

@ -0,0 +1,27 @@
# Streamingle Utilities
A Unity package containing reusable Streamingle utilities. The package is divided by feature while retaining the existing public namespaces for compatibility.
## Modules
- **Gaze** — Blendshape eye-gaze driver, Timeline track, guided calibration window, and tests.
- **Constraints** — Lightweight transform constraints and custom inspectors.
- **Camera**`SmoothFollow` component.
- **Timeline/Subtitles** — TextMeshPro subtitle Timeline track and SRT installer API.
- **Animation** — Blendshape gap-edge cleanup editor window.
## Editor menu conventions
- `Tools/Streamingle Utilities/Gaze/Calibration`
- `Tools/Streamingle Utilities/Animation/Blend Shape Gap Cleanup`
- `Component/Streamingle Utilities/Constraints/...`
- `Assets/Create/Streamingle Utilities/Gaze/Blendshape Gaze Profile`
## Dependencies
- Unity Timeline 1.8.10 or later
- TextMeshPro 5.0.0 or later
## Compatibility
The initial package release is validated against Unity 6 (`6000.3`). Public namespaces and the Gaze assembly names are retained during the package extraction to preserve existing project references.

7
README.md.meta Normal file
View File

@ -0,0 +1,7 @@
fileFormatVersion: 2
guid: a562d497ecca0904ea34d70550ba7b16
TextScriptImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

8
Runtime.meta Normal file
View File

@ -0,0 +1,8 @@
fileFormatVersion: 2
guid: 02520c32063675348b0277064ca5d534
folderAsset: yes
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

View File

@ -0,0 +1,152 @@
using UnityEngine;
namespace Streamingle.Utils
{
/// <summary>
/// 타겟 오브젝트를 부드럽게 따라가는 컴포넌트
/// </summary>
public class SmoothFollow : MonoBehaviour
{
[Header("Target")]
[Tooltip("따라갈 대상 Transform")]
public Transform target;
[Header("Follow Settings")]
[Tooltip("위치 따라가기 활성화")]
public bool followPosition = true;
[Tooltip("회전 따라가기 활성화")]
public bool followRotation = false;
[Header("Smoothing")]
[Tooltip("위치 보간 속도 (낮을수록 더 부드럽게)")]
[Range(0.01f, 20f)]
public float positionSmoothSpeed = 5f;
[Tooltip("회전 보간 속도 (낮을수록 더 부드럽게)")]
[Range(0.01f, 20f)]
public float rotationSmoothSpeed = 5f;
[Header("Offset")]
[Tooltip("타겟으로부터의 위치 오프셋 (로컬 기준)")]
public Vector3 positionOffset = Vector3.zero;
[Tooltip("타겟으로부터의 회전 오프셋")]
public Vector3 rotationOffset = Vector3.zero;
[Header("Constraints")]
[Tooltip("X축 위치 고정")]
public bool freezePositionX = false;
[Tooltip("Y축 위치 고정")]
public bool freezePositionY = false;
[Tooltip("Z축 위치 고정")]
public bool freezePositionZ = false;
private Vector3 _initialPosition;
private Quaternion _initialRotation;
private Vector3 _velocity = Vector3.zero;
private void Awake()
{
_initialPosition = transform.position;
_initialRotation = transform.rotation;
}
private void LateUpdate()
{
if (target == null) return;
if (followPosition)
{
UpdatePosition();
}
if (followRotation)
{
UpdateRotation();
}
}
private void UpdatePosition()
{
// 타겟 위치 + 로컬 오프셋 계산
Vector3 targetPosition = target.TransformPoint(positionOffset);
// 부드러운 보간
Vector3 smoothedPosition = Vector3.Lerp(
transform.position,
targetPosition,
positionSmoothSpeed * Time.deltaTime
);
// 축 고정 적용
if (freezePositionX) smoothedPosition.x = _initialPosition.x;
if (freezePositionY) smoothedPosition.y = _initialPosition.y;
if (freezePositionZ) smoothedPosition.z = _initialPosition.z;
transform.position = smoothedPosition;
}
private void UpdateRotation()
{
Quaternion targetRotation = target.rotation * Quaternion.Euler(rotationOffset);
transform.rotation = Quaternion.Slerp(
transform.rotation,
targetRotation,
rotationSmoothSpeed * Time.deltaTime
);
}
/// <summary>
/// 타겟을 런타임에 변경
/// </summary>
public void SetTarget(Transform newTarget)
{
target = newTarget;
}
/// <summary>
/// 즉시 타겟 위치로 이동 (보간 없이)
/// </summary>
public void SnapToTarget()
{
if (target == null) return;
if (followPosition)
{
Vector3 targetPosition = target.TransformPoint(positionOffset);
if (freezePositionX) targetPosition.x = _initialPosition.x;
if (freezePositionY) targetPosition.y = _initialPosition.y;
if (freezePositionZ) targetPosition.z = _initialPosition.z;
transform.position = targetPosition;
}
if (followRotation)
{
transform.rotation = target.rotation * Quaternion.Euler(rotationOffset);
}
}
/// <summary>
/// 초기 위치로 복귀
/// </summary>
public void ResetToInitial()
{
transform.position = _initialPosition;
transform.rotation = _initialRotation;
}
#if UNITY_EDITOR
private void OnDrawGizmosSelected()
{
if (target == null) return;
Gizmos.color = Color.cyan;
Vector3 targetPos = target.TransformPoint(positionOffset);
Gizmos.DrawLine(transform.position, targetPos);
Gizmos.DrawWireSphere(targetPos, 0.1f);
}
#endif
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: 53b895a2741ab0c4db91ec0dcb7c1a06

8
Runtime/Constraints.meta Normal file
View File

@ -0,0 +1,8 @@
fileFormatVersion: 2
guid: 31342348d6d2460382d40f327f52bedc
folderAsset: yes
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

View File

@ -0,0 +1,50 @@
using UnityEngine;
namespace Streamingle.Utils.RoughConstraints
{
[ExecuteAlways]
[AddComponentMenu("Streamingle Utilities/Constraints/Rough Aim Constraint")]
public class RoughAimConstraint : RoughConstraintBase
{
[Header("Aim")]
public Vector3 aimAxis = Vector3.forward;
public Vector3 upAxis = Vector3.up;
public Vector3 worldUpVector = Vector3.up;
public Vector3 rotationOffset;
protected override void ExecuteConstraint(float followT)
{
if (!TryGetWeightedPosition(out var targetPosition))
{
return;
}
var direction = targetPosition - transform.position;
if (direction.sqrMagnitude < 0.000001f)
{
return;
}
var targetRotation = LookRotationForAxis(direction, worldUpVector, aimAxis, upAxis);
targetRotation *= Quaternion.Euler(rotationOffset);
transform.rotation = Quaternion.Slerp(transform.rotation, targetRotation, followT * weight);
}
protected override void CaptureOffsetsFromCurrentPose()
{
if (!TryGetWeightedPosition(out var targetPosition))
{
return;
}
var direction = targetPosition - transform.position;
if (direction.sqrMagnitude < 0.000001f)
{
return;
}
var baseRotation = LookRotationForAxis(direction, worldUpVector, aimAxis, upAxis);
rotationOffset = (Quaternion.Inverse(baseRotation) * transform.rotation).eulerAngles;
}
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: 70b32d75aafa49d781fe9cd7389d78b5

View File

@ -0,0 +1,356 @@
using System.Collections.Generic;
using UnityEngine;
#if UNITY_EDITOR
using UnityEditor;
#endif
namespace Streamingle.Utils.RoughConstraints
{
public enum RoughConstraintUpdateMode
{
Update,
LateUpdate,
FixedUpdate,
Manual
}
[ExecuteAlways]
public abstract class RoughConstraintBase : MonoBehaviour
{
[Header("Constraint")]
public bool constraintActive = true;
public bool locked = true;
[Range(0f, 1f)]
public float weight = 1f;
public List<RoughConstraintSource> sources = new List<RoughConstraintSource>();
[Header("Rough Follow")]
[Min(0f)]
public float damping = 12f;
public RoughConstraintUpdateMode updateMode = RoughConstraintUpdateMode.LateUpdate;
public bool useUnscaledTime;
public bool snapOnEnable = true;
#if UNITY_EDITOR
private bool editorUpdateRegistered;
#endif
protected float CurrentDeltaTime
{
get
{
if (!Application.isPlaying)
{
return 1f / 60f;
}
if (updateMode == RoughConstraintUpdateMode.FixedUpdate)
{
return useUnscaledTime ? Time.fixedUnscaledDeltaTime : Time.fixedDeltaTime;
}
return useUnscaledTime ? Time.unscaledDeltaTime : Time.deltaTime;
}
}
protected float FollowT
{
get
{
if (damping <= 0f)
{
return 1f;
}
return 1f - Mathf.Exp(-damping * Mathf.Max(0f, CurrentDeltaTime));
}
}
protected virtual void OnEnable()
{
#if UNITY_EDITOR
RegisterEditorUpdate();
#endif
if (!snapOnEnable)
{
return;
}
SnapToSources();
}
protected virtual void OnDisable()
{
#if UNITY_EDITOR
UnregisterEditorUpdate();
#endif
}
private void Update()
{
if (!Application.isPlaying)
{
if (updateMode != RoughConstraintUpdateMode.Manual)
{
Evaluate();
}
return;
}
if (updateMode == RoughConstraintUpdateMode.Update)
{
Evaluate();
}
}
private void LateUpdate()
{
if (!Application.isPlaying)
{
return;
}
if (updateMode == RoughConstraintUpdateMode.LateUpdate)
{
Evaluate();
}
}
private void FixedUpdate()
{
if (!Application.isPlaying)
{
return;
}
if (updateMode == RoughConstraintUpdateMode.FixedUpdate)
{
Evaluate();
}
}
#if UNITY_EDITOR
private void RegisterEditorUpdate()
{
if (editorUpdateRegistered)
{
return;
}
EditorApplication.update += EditorUpdate;
editorUpdateRegistered = true;
}
private void UnregisterEditorUpdate()
{
if (!editorUpdateRegistered)
{
return;
}
EditorApplication.update -= EditorUpdate;
editorUpdateRegistered = false;
}
private void EditorUpdate()
{
if (this == null)
{
UnregisterEditorUpdate();
return;
}
if (Application.isPlaying || updateMode == RoughConstraintUpdateMode.Manual)
{
return;
}
Evaluate();
}
#endif
public void Evaluate()
{
if (!CanEvaluate())
{
return;
}
ExecuteConstraint(FollowT);
}
public void SnapToSources()
{
if (!CanEvaluate())
{
return;
}
ExecuteConstraint(1f);
}
public void SetLocked(bool value)
{
if (value && !locked)
{
CaptureOffsetsFromCurrentPose();
}
locked = value;
}
public void ActivateConstraint()
{
constraintActive = true;
locked = true;
SnapToSources();
}
protected abstract void ExecuteConstraint(float followT);
protected virtual void CaptureOffsetsFromCurrentPose()
{
}
private bool CanEvaluate()
{
return constraintActive && locked && weight > 0f;
}
protected bool TryGetWeightedPosition(out Vector3 position)
{
position = Vector3.zero;
var totalWeight = 0f;
for (var i = 0; i < sources.Count; i++)
{
var source = sources[i];
if (source.sourceTransform == null || source.weight <= 0f)
{
continue;
}
position += source.sourceTransform.position * source.weight;
totalWeight += source.weight;
}
if (totalWeight <= 0f)
{
return false;
}
position /= totalWeight;
return true;
}
protected bool TryGetWeightedRotation(out Quaternion rotation)
{
rotation = Quaternion.identity;
var hasRotation = false;
var accumulatedWeight = 0f;
for (var i = 0; i < sources.Count; i++)
{
var source = sources[i];
if (source.sourceTransform == null || source.weight <= 0f)
{
continue;
}
var sourceRotation = source.sourceTransform.rotation;
if (!hasRotation)
{
rotation = sourceRotation;
accumulatedWeight = source.weight;
hasRotation = true;
continue;
}
var lerpWeight = source.weight / (accumulatedWeight + source.weight);
rotation = Quaternion.Slerp(rotation, sourceRotation, lerpWeight);
accumulatedWeight += source.weight;
}
return hasRotation;
}
protected bool TryGetWeightedScale(out Vector3 scale)
{
scale = Vector3.zero;
var totalWeight = 0f;
for (var i = 0; i < sources.Count; i++)
{
var source = sources[i];
if (source.sourceTransform == null || source.weight <= 0f)
{
continue;
}
scale += source.sourceTransform.localScale * source.weight;
totalWeight += source.weight;
}
if (totalWeight <= 0f)
{
return false;
}
scale /= totalWeight;
return true;
}
protected static Vector3 ApplyAxisMask(Vector3 current, Vector3 target, Vector3Bool constrainedAxis)
{
if (!constrainedAxis.x) target.x = current.x;
if (!constrainedAxis.y) target.y = current.y;
if (!constrainedAxis.z) target.z = current.z;
return target;
}
protected static Vector3 ApplyEulerAxisMask(Quaternion current, Quaternion target, Vector3Bool constrainedAxis)
{
var currentEuler = current.eulerAngles;
var targetEuler = target.eulerAngles;
if (!constrainedAxis.x) targetEuler.x = currentEuler.x;
if (!constrainedAxis.y) targetEuler.y = currentEuler.y;
if (!constrainedAxis.z) targetEuler.z = currentEuler.z;
return targetEuler;
}
protected static Quaternion LookRotationForAxis(Vector3 direction, Vector3 worldUp, Vector3 localForward, Vector3 localUp)
{
var safeDirection = SafeNormal(direction, Vector3.forward);
var safeWorldUp = SafeUp(safeDirection, worldUp);
var safeLocalForward = SafeNormal(localForward, Vector3.forward);
var safeLocalUp = SafeUp(safeLocalForward, localUp);
var targetWorldRotation = Quaternion.LookRotation(safeDirection, safeWorldUp);
var localAxisRotation = Quaternion.LookRotation(safeLocalForward, safeLocalUp);
return targetWorldRotation * Quaternion.Inverse(localAxisRotation);
}
private static Vector3 SafeNormal(Vector3 value, Vector3 fallback)
{
return value.sqrMagnitude > 0.000001f ? value.normalized : fallback;
}
private static Vector3 SafeUp(Vector3 forward, Vector3 up)
{
var safeUp = SafeNormal(up, Vector3.up);
if (Mathf.Abs(Vector3.Dot(forward, safeUp)) < 0.999f)
{
return safeUp;
}
return Mathf.Abs(Vector3.Dot(forward, Vector3.up)) < 0.999f ? Vector3.up : Vector3.right;
}
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: 113d932d171c415fa3b62ab75bdf3de7

View File

@ -0,0 +1,14 @@
using System;
using UnityEngine;
namespace Streamingle.Utils.RoughConstraints
{
[Serializable]
public struct RoughConstraintSource
{
public Transform sourceTransform;
[Range(0f, 1f)]
public float weight;
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: 7192f25e37a54bc385855e11d7f3abc3

View File

@ -0,0 +1,52 @@
using UnityEngine;
namespace Streamingle.Utils.RoughConstraints
{
[ExecuteAlways]
[AddComponentMenu("Streamingle Utilities/Constraints/Rough Look At Constraint")]
public class RoughLookAtConstraint : RoughConstraintBase
{
[Header("Look At")]
public Vector3 forwardAxis = Vector3.forward;
public Vector3 worldUpVector = Vector3.up;
public Vector3 targetPositionOffset;
public Vector3 rotationOffset;
protected override void ExecuteConstraint(float followT)
{
if (!TryGetWeightedPosition(out var targetPosition))
{
return;
}
targetPosition += targetPositionOffset;
var direction = targetPosition - transform.position;
if (direction.sqrMagnitude < 0.000001f)
{
return;
}
var targetRotation = LookRotationForAxis(direction, worldUpVector, forwardAxis, Vector3.up);
targetRotation *= Quaternion.Euler(rotationOffset);
transform.rotation = Quaternion.Slerp(transform.rotation, targetRotation, followT * weight);
}
protected override void CaptureOffsetsFromCurrentPose()
{
if (!TryGetWeightedPosition(out var targetPosition))
{
return;
}
targetPosition += targetPositionOffset;
var direction = targetPosition - transform.position;
if (direction.sqrMagnitude < 0.000001f)
{
return;
}
var baseRotation = LookRotationForAxis(direction, worldUpVector, forwardAxis, Vector3.up);
rotationOffset = (Quaternion.Inverse(baseRotation) * transform.rotation).eulerAngles;
}
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: 0ff851637cd645758f43435b7ee8fb42

View File

@ -0,0 +1,67 @@
using UnityEngine;
namespace Streamingle.Utils.RoughConstraints
{
[ExecuteAlways]
[AddComponentMenu("Streamingle Utilities/Constraints/Rough Parent Constraint")]
public class RoughParentConstraint : RoughConstraintBase
{
[Header("Position")]
public Vector3Bool constrainedPositionAxis = Vector3Bool.All;
public Vector3 positionOffset;
public bool positionOffsetInSourceSpace = true;
[Header("Rotation")]
public Vector3Bool constrainedRotationAxis = Vector3Bool.All;
public Vector3 rotationOffset;
protected override void ExecuteConstraint(float followT)
{
var blend = followT * weight;
if (TryGetWeightedPosition(out var targetPosition))
{
if (positionOffsetInSourceSpace && sources.Count > 0 && sources[0].sourceTransform != null)
{
targetPosition += sources[0].sourceTransform.TransformVector(positionOffset);
}
else
{
targetPosition += positionOffset;
}
targetPosition = ApplyAxisMask(transform.position, targetPosition, constrainedPositionAxis);
transform.position = Vector3.Lerp(transform.position, targetPosition, blend);
}
if (TryGetWeightedRotation(out var targetRotation))
{
targetRotation *= Quaternion.Euler(rotationOffset);
var maskedEuler = ApplyEulerAxisMask(transform.rotation, targetRotation, constrainedRotationAxis);
targetRotation = Quaternion.Euler(maskedEuler);
transform.rotation = Quaternion.Slerp(transform.rotation, targetRotation, blend);
}
}
protected override void CaptureOffsetsFromCurrentPose()
{
if (TryGetWeightedPosition(out var sourcePosition))
{
var delta = transform.position - sourcePosition;
if (positionOffsetInSourceSpace && sources.Count > 0 && sources[0].sourceTransform != null)
{
positionOffset = sources[0].sourceTransform.InverseTransformVector(delta);
}
else
{
positionOffset = delta;
}
}
if (TryGetWeightedRotation(out var sourceRotation))
{
rotationOffset = (Quaternion.Inverse(sourceRotation) * transform.rotation).eulerAngles;
}
}
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: 015afbd1174749d889a7b3542929c39b

View File

@ -0,0 +1,52 @@
using UnityEngine;
namespace Streamingle.Utils.RoughConstraints
{
[ExecuteAlways]
[AddComponentMenu("Streamingle Utilities/Constraints/Rough Position Constraint")]
public class RoughPositionConstraint : RoughConstraintBase
{
[Header("Position")]
public Vector3Bool constrainedAxis = Vector3Bool.All;
public Vector3 positionOffset;
public bool offsetInSourceSpace = true;
protected override void ExecuteConstraint(float followT)
{
if (!TryGetWeightedPosition(out var targetPosition))
{
return;
}
if (offsetInSourceSpace && sources.Count > 0 && sources[0].sourceTransform != null)
{
targetPosition += sources[0].sourceTransform.TransformVector(positionOffset);
}
else
{
targetPosition += positionOffset;
}
targetPosition = ApplyAxisMask(transform.position, targetPosition, constrainedAxis);
transform.position = Vector3.Lerp(transform.position, targetPosition, followT * weight);
}
protected override void CaptureOffsetsFromCurrentPose()
{
if (!TryGetWeightedPosition(out var sourcePosition))
{
return;
}
var delta = transform.position - sourcePosition;
if (offsetInSourceSpace && sources.Count > 0 && sources[0].sourceTransform != null)
{
positionOffset = sources[0].sourceTransform.InverseTransformVector(delta);
}
else
{
positionOffset = delta;
}
}
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: b1874b6468e0419f925ee085ce478bfb

View File

@ -0,0 +1,36 @@
using UnityEngine;
namespace Streamingle.Utils.RoughConstraints
{
[ExecuteAlways]
[AddComponentMenu("Streamingle Utilities/Constraints/Rough Rotation Constraint")]
public class RoughRotationConstraint : RoughConstraintBase
{
[Header("Rotation")]
public Vector3Bool constrainedAxis = Vector3Bool.All;
public Vector3 rotationOffset;
protected override void ExecuteConstraint(float followT)
{
if (!TryGetWeightedRotation(out var targetRotation))
{
return;
}
targetRotation *= Quaternion.Euler(rotationOffset);
var maskedEuler = ApplyEulerAxisMask(transform.rotation, targetRotation, constrainedAxis);
targetRotation = Quaternion.Euler(maskedEuler);
transform.rotation = Quaternion.Slerp(transform.rotation, targetRotation, followT * weight);
}
protected override void CaptureOffsetsFromCurrentPose()
{
if (!TryGetWeightedRotation(out var sourceRotation))
{
return;
}
rotationOffset = (Quaternion.Inverse(sourceRotation) * transform.rotation).eulerAngles;
}
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: 23820ffe249c4b7bad25455ca97fc8d3

View File

@ -0,0 +1,35 @@
using UnityEngine;
namespace Streamingle.Utils.RoughConstraints
{
[ExecuteAlways]
[AddComponentMenu("Streamingle Utilities/Constraints/Rough Scale Constraint")]
public class RoughScaleConstraint : RoughConstraintBase
{
[Header("Scale")]
public Vector3Bool constrainedAxis = Vector3Bool.All;
public Vector3 scaleOffset;
protected override void ExecuteConstraint(float followT)
{
if (!TryGetWeightedScale(out var targetScale))
{
return;
}
targetScale += scaleOffset;
targetScale = ApplyAxisMask(transform.localScale, targetScale, constrainedAxis);
transform.localScale = Vector3.Lerp(transform.localScale, targetScale, followT * weight);
}
protected override void CaptureOffsetsFromCurrentPose()
{
if (!TryGetWeightedScale(out var sourceScale))
{
return;
}
scaleOffset = transform.localScale - sourceScale;
}
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: c58bfc39585a4c61bef1c73bf3fbc9a0

View File

@ -0,0 +1,21 @@
using System;
namespace Streamingle.Utils.RoughConstraints
{
[Serializable]
public struct Vector3Bool
{
public bool x;
public bool y;
public bool z;
public Vector3Bool(bool x, bool y, bool z)
{
this.x = x;
this.y = y;
this.z = z;
}
public static Vector3Bool All => new Vector3Bool(true, true, true);
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: 6b3fce5b0afa436abaaba5ea0261354d

8
Runtime/Gaze.meta Normal file
View File

@ -0,0 +1,8 @@
fileFormatVersion: 2
guid: 6eb0f0d07a0da5d49b8011fd6f2acbfb
folderAsset: yes
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

View File

@ -0,0 +1,749 @@
using System;
using System.Collections.Generic;
using System.Runtime.CompilerServices;
using UnityEngine;
[assembly: InternalsVisibleTo("Streamingle.Gaze.Editor")]
[assembly: InternalsVisibleTo("Streamingle.Gaze.Tests.Editor")]
namespace Streamingle.Gaze
{
[ExecuteAlways]
[DisallowMultipleComponent]
[DefaultExecutionOrder(10000)]
public sealed class BlendshapeGazeDriver : MonoBehaviour
{
private const float MinimumInfluence = 0.0001f;
private const float ExternalWriteTolerance = 0.001f;
[Header("Required")]
[SerializeField]
private SkinnedMeshRenderer targetRenderer;
[SerializeField]
private BlendshapeGazeProfile profile;
[SerializeField]
[Tooltip("Eye midpoint. Its position is used as the gaze origin.")]
private Transform gazeOrigin;
[SerializeField]
[Tooltip("Optional forward/up fallback when Gaze Origin is not assigned. Gaze Origin rotation is authoritative when present.")]
private Transform headReference;
[Header("Default Target")]
[SerializeField]
[Tooltip("Optional output camera override. Timeline clips with no target use this before Camera.main.")]
private Camera targetCamera;
[SerializeField]
[Tooltip("Optional fallback target when Timeline is not driving this component.")]
private Transform defaultTarget;
[SerializeField]
private bool defaultToMainCamera;
[SerializeField, Range(0f, 1f)]
private float defaultInfluence;
[SerializeField]
private bool defaultCorrectives;
[Header("Camera Cut")]
[SerializeField]
private GazeCutResponse cutResponse = GazeCutResponse.Snap;
[SerializeField, Min(0f)]
private float cutTransitionDuration = 0.1f;
[SerializeField, Range(0f, 180f)]
[Tooltip("A target-direction jump at or above this angle is treated as a camera cut. Set 0 to disable angle detection.")]
private float cutDetectionAngle = 15f;
[Header("Debug Gizmos")]
[SerializeField]
private bool drawGizmos = true;
[SerializeField, Min(0.01f)]
private float gizmoDistance = 0.35f;
private readonly Dictionary<int, GazeTimelineState> timelineStates =
new Dictionary<int, GazeTimelineState>();
private int[] blendShapeIndices = Array.Empty<int>();
private float[] baselineWeights = Array.Empty<float>();
private float[] evaluatedWeights = Array.Empty<float>();
private float[] lastAppliedWeights = Array.Empty<float>();
private string[] cachedChannelNames = Array.Empty<string>();
private SkinnedMeshRenderer cachedRenderer;
private Mesh cachedMesh;
private BlendshapeGazeProfile cachedProfile;
private bool hasAppliedWeights;
private bool hasRuntimeState;
private GazeTimelineState runtimeState;
private bool hasPreviousDirection;
private Vector3 previousDesiredDirection;
private UnityEngine.Object previousTargetIdentity;
private bool cutTransitionActive;
private Vector3 cutTransitionFromDirection;
private float cutTransitionStartTime;
private int calibrationPreviewOverrideCount;
public SkinnedMeshRenderer TargetRenderer
{
get => targetRenderer;
set
{
if (targetRenderer == value)
return;
ReleaseWeights();
targetRenderer = value;
InvalidateCache();
}
}
public BlendshapeGazeProfile Profile
{
get => profile;
set
{
if (profile == value)
return;
ReleaseWeights();
profile = value;
InvalidateCache();
}
}
public Transform GazeOrigin
{
get => gazeOrigin;
set => gazeOrigin = value;
}
public Transform HeadReference
{
get => headReference;
set => headReference = value;
}
public Camera TargetCamera
{
get => targetCamera;
set => targetCamera = value;
}
public GazeDirectionResult LastDirection { get; private set; }
public Vector3 LastOriginPosition { get; private set; }
public Vector3 LastTargetPosition { get; private set; }
public bool HasActiveRequest { get; private set; }
public Vector3 GetGazeOriginPosition()
{
if (gazeOrigin != null)
return gazeOrigin.position;
if (targetRenderer != null)
return targetRenderer.bounds.center;
return transform.position;
}
public bool RebuildCache()
{
ReleaseWeights();
InvalidateCache();
return EnsureCache();
}
private Transform DirectionReference => gazeOrigin != null
? gazeOrigin
: (headReference != null ? headReference : transform);
private void LateUpdate()
{
if (Application.IsPlaying(gameObject))
{
EvaluateNow(false);
return;
}
#if UNITY_EDITOR
// ExecuteAlways receives the editor player loop requested by the
// Timeline mixer. This is a stronger final ordering point than a
// one-shot delayCall: Director animation has already updated the Head,
// GazeOrigin and facial blendshapes before this high-order LateUpdate.
if (!Application.isPlaying && timelineStates.Count > 0)
EvaluateNow(true);
#endif
}
public void SetTimelineState(int sourceId, GazeTimelineState state)
{
state.Influence = SanitizeInfluence(state.Influence);
state.CutTransitionDuration = SanitizeNonNegative(state.CutTransitionDuration);
timelineStates[sourceId] = state;
}
public void ClearTimelineState(int sourceId)
{
timelineStates.Remove(sourceId);
if (timelineStates.Count == 0)
{
HasActiveRequest = false;
ReleaseWeights();
ResetDirectionHistory();
}
}
public void SetRuntimeTarget(
Transform target,
float influence = 1f,
bool enableCorrectives = false)
{
runtimeState = new GazeTimelineState
{
Target = target,
UseMainCamera = target == null,
Influence = SanitizeInfluence(influence),
EnableCorrectives = enableCorrectives,
ResetSmoothing = !hasRuntimeState
};
hasRuntimeState = true;
}
public void SetRuntimeTargetPosition(
Vector3 worldPosition,
float influence = 1f,
bool enableCorrectives = false)
{
runtimeState = new GazeTimelineState
{
TargetPosition = worldPosition,
HasTargetPosition = true,
Influence = SanitizeInfluence(influence),
EnableCorrectives = enableCorrectives,
ResetSmoothing = !hasRuntimeState
};
hasRuntimeState = true;
}
public void ClearRuntimeTarget()
{
hasRuntimeState = false;
if (timelineStates.Count == 0)
{
HasActiveRequest = false;
ReleaseWeights();
ResetDirectionHistory();
}
}
public void EvaluateNow(bool deterministic)
{
// Calibration owns the configured blendshape channels while its preview
// session is active. Timeline/editor evaluation can still update its
// request state, but must not overwrite (or restore over) that preview.
if (calibrationPreviewOverrideCount > 0)
{
HasActiveRequest = false;
return;
}
if (!TryGetActiveState(out var state, out var usesRuntimeState))
{
HasActiveRequest = false;
ReleaseWeights();
ResetDirectionHistory();
return;
}
state.Deterministic |= deterministic;
if (!TryResolveTarget(state, out var targetPosition, out var targetIdentity))
{
HasActiveRequest = false;
ReleaseWeights();
ResetDirectionHistory();
return;
}
if (!EnsureCache())
{
HasActiveRequest = false;
ReleaseWeights();
return;
}
LastOriginPosition = GetGazeOriginPosition();
LastTargetPosition = targetPosition;
var desiredDirection = targetPosition - LastOriginPosition;
if (desiredDirection.sqrMagnitude <= Mathf.Epsilon)
{
HasActiveRequest = false;
ReleaseWeights();
return;
}
desiredDirection.Normalize();
var effectiveDirection = ApplyCutResponse(
desiredDirection,
targetIdentity,
state,
state.Deterministic || !Application.isPlaying);
var reference = DirectionReference;
LastDirection = GazeDirectionSolver.Calculate(
reference != null ? reference.rotation : Quaternion.identity,
effectiveDirection,
profile.MinYaw,
profile.MaxYaw,
profile.MinPitch,
profile.MaxPitch);
if (!profile.TryEvaluate(LastDirection, state.EnableCorrectives, evaluatedWeights))
{
HasActiveRequest = false;
ReleaseWeights();
return;
}
CaptureExternalBaseline();
ApplyWeights(Mathf.Clamp01(state.Influence));
HasActiveRequest = true;
if (usesRuntimeState)
runtimeState.ResetSmoothing = false;
}
internal void BeginCalibrationPreviewOverride()
{
if (calibrationPreviewOverrideCount == 0)
{
// If gaze was already driving the renderer, return those channels to
// their external facial baseline before the preview captures it.
ReleaseWeights();
ResetDirectionHistory();
HasActiveRequest = false;
}
calibrationPreviewOverrideCount++;
}
internal void EndCalibrationPreviewOverride()
{
if (calibrationPreviewOverrideCount <= 0)
return;
calibrationPreviewOverrideCount--;
if (calibrationPreviewOverrideCount == 0)
ResetDirectionHistory();
}
internal void PrepareForTimelineEvaluation()
{
if (!isActiveAndEnabled || calibrationPreviewOverrideCount > 0)
return;
// PlayableGraph PrepareFrame runs before Timeline animation outputs.
// Restore the prior facial baseline here, then LateUpdate/editor final
// evaluation can capture the newly sampled value even when it happens
// to be numerically identical to our previous gaze result.
ReleaseWeights();
}
private bool TryGetActiveState(out GazeTimelineState state, out bool usesRuntimeState)
{
var found = false;
var bestInfluence = MinimumInfluence;
var bestSourceId = int.MaxValue;
state = default;
usesRuntimeState = false;
foreach (var pair in timelineStates)
{
var candidate = pair.Value;
if (candidate.Influence <= MinimumInfluence)
continue;
if (found && candidate.Influence < bestInfluence)
continue;
if (found && Mathf.Approximately(candidate.Influence, bestInfluence)
&& pair.Key >= bestSourceId)
continue;
state = candidate;
bestInfluence = candidate.Influence;
bestSourceId = pair.Key;
found = true;
}
if (found)
return true;
if (hasRuntimeState && runtimeState.Influence > MinimumInfluence)
{
state = runtimeState;
usesRuntimeState = true;
return true;
}
if (defaultInfluence <= MinimumInfluence || (defaultTarget == null && !defaultToMainCamera))
return false;
state = new GazeTimelineState
{
Target = defaultTarget,
UseMainCamera = defaultTarget == null && defaultToMainCamera,
Influence = defaultInfluence,
EnableCorrectives = defaultCorrectives
};
return true;
}
private bool TryResolveTarget(
GazeTimelineState state,
out Vector3 targetPosition,
out UnityEngine.Object identity)
{
if (state.HasTargetPosition)
{
targetPosition = state.TargetPosition;
identity = null;
return IsFinite(targetPosition);
}
if (state.HasTargetDirection)
{
var targetDirection = state.TargetDirection;
if (!IsFinite(targetDirection) || targetDirection.sqrMagnitude <= Mathf.Epsilon)
{
targetPosition = default;
identity = null;
return false;
}
targetPosition = GetGazeOriginPosition() + targetDirection.normalized;
identity = null;
return true;
}
var target = state.Target;
if (target == null && state.UseMainCamera)
{
var mainCamera = targetCamera != null ? targetCamera : Camera.main;
target = mainCamera != null ? mainCamera.transform : null;
}
if (target == null)
{
targetPosition = default;
identity = null;
return false;
}
targetPosition = target.position;
identity = target;
return IsFinite(targetPosition);
}
private Vector3 ApplyCutResponse(
Vector3 desiredDirection,
UnityEngine.Object targetIdentity,
GazeTimelineState state,
bool deterministic)
{
var response = state.OverrideCutResponse ? state.CutResponse : cutResponse;
var duration = state.OverrideCutResponse
? state.CutTransitionDuration
: cutTransitionDuration;
if (deterministic || state.ResetSmoothing || !hasPreviousDirection)
{
previousDesiredDirection = desiredDirection;
previousTargetIdentity = targetIdentity;
hasPreviousDirection = true;
cutTransitionActive = false;
return desiredDirection;
}
var identityChanged = previousTargetIdentity != targetIdentity;
// Zero disables pose-jump detection while still allowing an explicit target
// identity change to count as a cut.
var angleJump = cutDetectionAngle > 0f
&& Vector3.Angle(previousDesiredDirection, desiredDirection) >= cutDetectionAngle;
if (response == GazeCutResponse.Smooth && duration > Mathf.Epsilon
&& (identityChanged || angleJump && !cutTransitionActive))
{
cutTransitionFromDirection = cutTransitionActive
? EvaluateCutTransition(previousDesiredDirection, duration)
: previousDesiredDirection;
cutTransitionStartTime = Time.unscaledTime;
cutTransitionActive = true;
}
previousDesiredDirection = desiredDirection;
previousTargetIdentity = targetIdentity;
if (response != GazeCutResponse.Smooth || duration <= Mathf.Epsilon)
{
cutTransitionActive = false;
return desiredDirection;
}
return EvaluateCutTransition(desiredDirection, duration);
}
private Vector3 EvaluateCutTransition(Vector3 destination, float duration)
{
if (!cutTransitionActive)
return destination;
var t = Mathf.Clamp01((Time.unscaledTime - cutTransitionStartTime) / duration);
if (t >= 1f)
{
cutTransitionActive = false;
return destination;
}
return Vector3.Slerp(cutTransitionFromDirection, destination, t).normalized;
}
private bool EnsureCache()
{
var mesh = targetRenderer != null ? targetRenderer.sharedMesh : null;
if (targetRenderer == null || mesh == null || profile == null || profile.Channels.Count == 0
|| !profile.IsCompatibleWith(mesh))
return false;
if (cachedRenderer == targetRenderer && cachedMesh == mesh && cachedProfile == profile
&& blendShapeIndices.Length == profile.Channels.Count && ChannelMappingMatches())
return true;
ReleaseWeights();
cachedRenderer = targetRenderer;
cachedMesh = mesh;
cachedProfile = profile;
var count = profile.Channels.Count;
blendShapeIndices = new int[count];
baselineWeights = new float[count];
evaluatedWeights = new float[count];
lastAppliedWeights = new float[count];
cachedChannelNames = new string[count];
var resolvedIndices = new HashSet<int>();
for (var i = 0; i < count; i++)
{
cachedChannelNames[i] = profile.Channels[i].BlendShapeName;
blendShapeIndices[i] = profile.FindBlendShapeIndex(mesh, i);
if (blendShapeIndices[i] < 0 || !resolvedIndices.Add(blendShapeIndices[i]))
{
InvalidateCache();
return false;
}
}
return true;
}
private void CaptureExternalBaseline()
{
for (var i = 0; i < blendShapeIndices.Length; i++)
{
var current = targetRenderer.GetBlendShapeWeight(blendShapeIndices[i]);
if (!hasAppliedWeights || Mathf.Abs(current - lastAppliedWeights[i]) > ExternalWriteTolerance)
baselineWeights[i] = current;
}
}
private void ApplyWeights(float influence)
{
for (var i = 0; i < blendShapeIndices.Length; i++)
{
var targetWeight = evaluatedWeights[i];
var channel = profile.Channels[i];
float result;
if (float.IsNaN(targetWeight))
{
result = baselineWeights[i];
}
else if (channel.Usage == GazeChannelUsage.Corrective
&& channel.CorrectiveBlendMode == GazeCorrectiveBlendMode.AdditiveFromCenter)
{
var centerWeight = profile.GetSampleWeight(GazeCalibrationPoint.Center, i);
result = baselineWeights[i] + (targetWeight - centerWeight) * influence;
}
else
{
result = Mathf.LerpUnclamped(baselineWeights[i], targetWeight, influence);
}
targetRenderer.SetBlendShapeWeight(blendShapeIndices[i], result);
lastAppliedWeights[i] = result;
}
hasAppliedWeights = true;
}
private bool ChannelMappingMatches()
{
if (cachedChannelNames.Length != profile.Channels.Count)
return false;
for (var i = 0; i < cachedChannelNames.Length; i++)
{
var channel = profile.Channels[i];
if (channel == null || !string.Equals(
cachedChannelNames[i],
channel.BlendShapeName,
StringComparison.Ordinal))
return false;
}
return true;
}
private void ReleaseWeights()
{
if (!hasAppliedWeights || cachedRenderer == null)
return;
// The old indices are only meaningful for the mesh they were built from.
if (cachedRenderer.sharedMesh != cachedMesh)
{
hasAppliedWeights = false;
return;
}
// If Animator/Timeline/a live receiver has written since our previous LateUpdate,
// that current value is the new base. Otherwise restore the last base so stopping
// the gaze track never leaves its final pose stuck on the face.
for (var i = 0; i < blendShapeIndices.Length && i < baselineWeights.Length; i++)
{
var index = blendShapeIndices[i];
if (index < 0)
continue;
var current = cachedRenderer.GetBlendShapeWeight(index);
if (Mathf.Abs(current - lastAppliedWeights[i]) > ExternalWriteTolerance)
baselineWeights[i] = current;
cachedRenderer.SetBlendShapeWeight(index, baselineWeights[i]);
}
hasAppliedWeights = false;
}
private void ResetDirectionHistory()
{
hasPreviousDirection = false;
previousTargetIdentity = null;
cutTransitionActive = false;
}
private void InvalidateCache()
{
cachedRenderer = null;
cachedMesh = null;
cachedProfile = null;
blendShapeIndices = Array.Empty<int>();
baselineWeights = Array.Empty<float>();
evaluatedWeights = Array.Empty<float>();
lastAppliedWeights = Array.Empty<float>();
cachedChannelNames = Array.Empty<string>();
hasAppliedWeights = false;
}
private void OnDisable()
{
#if UNITY_EDITOR
BlendshapeGazeMixerBehaviour.CancelEditorFinalEvaluation(this);
#endif
ReleaseWeights();
timelineStates.Clear();
ResetDirectionHistory();
HasActiveRequest = false;
}
private void OnValidate()
{
defaultInfluence = SanitizeInfluence(defaultInfluence);
cutTransitionDuration = SanitizeNonNegative(cutTransitionDuration);
cutDetectionAngle = IsFinite(cutDetectionAngle)
? Mathf.Clamp(cutDetectionAngle, 0f, 180f)
: 15f;
gizmoDistance = IsFinite(gizmoDistance)
? Mathf.Max(0.01f, gizmoDistance)
: 0.35f;
ReleaseWeights();
InvalidateCache();
}
private void OnDrawGizmos()
{
if (!drawGizmos || profile == null)
return;
var origin = gazeOrigin != null
? gazeOrigin.position
: (targetRenderer != null ? targetRenderer.bounds.center : transform.position);
var reference = DirectionReference;
var rotation = reference != null ? reference.rotation : Quaternion.identity;
var distance = Mathf.Max(0.01f, gizmoDistance);
var downLeft = origin + rotation * GazeDirectionSolver.DirectionFromAngles(
profile.MinYaw,
profile.MinPitch) * distance;
var downRight = origin + rotation * GazeDirectionSolver.DirectionFromAngles(
profile.MaxYaw,
profile.MinPitch) * distance;
var upLeft = origin + rotation * GazeDirectionSolver.DirectionFromAngles(
profile.MinYaw,
profile.MaxPitch) * distance;
var upRight = origin + rotation * GazeDirectionSolver.DirectionFromAngles(
profile.MaxYaw,
profile.MaxPitch) * distance;
Gizmos.color = new Color(0.2f, 0.9f, 1f, 0.8f);
Gizmos.DrawWireSphere(origin, distance * 0.025f);
Gizmos.DrawLine(origin, downLeft);
Gizmos.DrawLine(origin, downRight);
Gizmos.DrawLine(origin, upLeft);
Gizmos.DrawLine(origin, upRight);
Gizmos.DrawLine(downLeft, downRight);
Gizmos.DrawLine(downRight, upRight);
Gizmos.DrawLine(upRight, upLeft);
Gizmos.DrawLine(upLeft, downLeft);
Gizmos.color = Color.green;
Gizmos.DrawLine(origin, origin + rotation * Vector3.forward * distance);
if (!HasActiveRequest)
return;
Gizmos.color = LastDirection.WasClamped ? Color.red : Color.cyan;
Gizmos.DrawLine(origin, LastTargetPosition);
Gizmos.DrawWireSphere(LastTargetPosition, distance * 0.035f);
var clampedDirection = rotation * GazeDirectionSolver.DirectionFromAngles(
LastDirection.ClampedYaw,
LastDirection.ClampedPitch);
Gizmos.color = Color.yellow;
Gizmos.DrawLine(origin, origin + clampedDirection * distance);
}
private static bool IsFinite(Vector3 value)
{
return !float.IsNaN(value.x) && !float.IsInfinity(value.x)
&& !float.IsNaN(value.y) && !float.IsInfinity(value.y)
&& !float.IsNaN(value.z) && !float.IsInfinity(value.z);
}
private static bool IsFinite(float value)
{
return !float.IsNaN(value) && !float.IsInfinity(value);
}
private static float SanitizeInfluence(float value)
{
return IsFinite(value) ? Mathf.Clamp01(value) : 0f;
}
private static float SanitizeNonNegative(float value)
{
return IsFinite(value) ? Mathf.Max(0f, value) : 0f;
}
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: 21992d28728ca584d85dd4fbbb72b02b

View File

@ -0,0 +1,634 @@
using System;
using System.Collections.Generic;
using System.Text;
using UnityEngine;
namespace Streamingle.Gaze
{
[CreateAssetMenu(fileName = "BlendshapeGazeProfile", menuName = "Streamingle Utilities/Gaze/Blendshape Gaze Profile")]
public sealed class BlendshapeGazeProfile : ScriptableObject
{
private const float DefaultMinYaw = -35f;
private const float DefaultMaxYaw = 35f;
private const float DefaultMinPitch = -25f;
private const float DefaultMaxPitch = 20f;
private static readonly string[] StandardEyeLookNames =
{
"eyeLookUpLeft",
"eyeLookUpRight",
"eyeLookDownLeft",
"eyeLookDownRight",
"eyeLookInLeft",
"eyeLookInRight",
"eyeLookOutLeft",
"eyeLookOutRight"
};
[Header("Direction Limits")]
[SerializeField, Range(-90f, -0.1f)]
private float minYaw = DefaultMinYaw;
[SerializeField, Range(0.1f, 90f)]
private float maxYaw = DefaultMaxYaw;
[SerializeField, Range(-90f, -0.1f)]
private float minPitch = DefaultMinPitch;
[SerializeField, Range(0.1f, 90f)]
private float maxPitch = DefaultMaxPitch;
[Header("Source Mesh")]
[SerializeField]
[Tooltip("Optional calibration mesh guard. Clear this only when intentionally sharing a profile across compatible meshes.")]
private Mesh sourceMesh;
[Header("Blendshape Mapping")]
[SerializeField]
private List<GazeBlendShapeChannel> channels = new List<GazeBlendShapeChannel>();
[Header("3x3 Calibration")]
[SerializeField]
private List<GazeCalibrationSample> samples = new List<GazeCalibrationSample>(9);
[SerializeField, HideInInspector]
private string channelLayoutSignature;
public float MinYaw => minYaw;
public float MaxYaw => maxYaw;
public float MinPitch => minPitch;
public float MaxPitch => maxPitch;
public Mesh SourceMesh => sourceMesh;
public IReadOnlyList<GazeBlendShapeChannel> Channels => channels;
public IReadOnlyList<GazeCalibrationSample> Samples => samples;
public bool IsFullyCalibrated
{
get
{
EnsureGrid();
if (channels.Count == 0)
return false;
for (var i = 0; i < samples.Count; i++)
{
if (samples[i] == null || !samples[i].Captured
|| samples[i].Weights.Count != channels.Count)
return false;
}
return true;
}
}
public bool IsSampleCaptured(GazeCalibrationPoint point)
{
EnsureGrid();
var sampleIndex = (int)point;
return sampleIndex >= 0 && sampleIndex < samples.Count
&& samples[sampleIndex] != null
&& samples[sampleIndex].Captured
&& samples[sampleIndex].Weights.Count == channels.Count;
}
public float[] GetSampleWeightsCopy(GazeCalibrationPoint point)
{
EnsureGrid();
var sampleIndex = (int)point;
if (sampleIndex < 0 || sampleIndex >= samples.Count)
return Array.Empty<float>();
var copy = new float[channels.Count];
var sample = samples[sampleIndex];
if (sample == null)
return copy;
for (var i = 0; i < copy.Length; i++)
copy[i] = sample.GetWeight(i);
return copy;
}
public bool CanGenerateCorners
{
get
{
return channels.Count > 0
&& IsSampleCaptured(GazeCalibrationPoint.Center)
&& IsSampleCaptured(GazeCalibrationPoint.CenterLeft)
&& IsSampleCaptured(GazeCalibrationPoint.CenterRight)
&& IsSampleCaptured(GazeCalibrationPoint.UpCenter)
&& IsSampleCaptured(GazeCalibrationPoint.DownCenter);
}
}
public bool GenerateCornersFromCardinals()
{
if (!CanGenerateCorners)
return false;
var center = GetSampleWeightsCopy(GazeCalibrationPoint.Center);
var left = GetSampleWeightsCopy(GazeCalibrationPoint.CenterLeft);
var right = GetSampleWeightsCopy(GazeCalibrationPoint.CenterRight);
var up = GetSampleWeightsCopy(GazeCalibrationPoint.UpCenter);
var down = GetSampleWeightsCopy(GazeCalibrationPoint.DownCenter);
SetGeneratedCorner(GazeCalibrationPoint.DownLeft, down, left, center);
SetGeneratedCorner(GazeCalibrationPoint.DownRight, down, right, center);
SetGeneratedCorner(GazeCalibrationPoint.UpLeft, up, left, center);
SetGeneratedCorner(GazeCalibrationPoint.UpRight, up, right, center);
return true;
}
public bool SeedStandardEyeLook(float maximumWeight = 100f)
{
if (channels.Count == 0 || float.IsNaN(maximumWeight) || float.IsInfinity(maximumWeight))
return false;
var standardIndices = new int[StandardEyeLookNames.Length];
for (var i = 0; i < StandardEyeLookNames.Length; i++)
{
standardIndices[i] = FindChannelIndexByNormalizedSuffix(StandardEyeLookNames[i]);
if (standardIndices[i] < 0)
return false;
}
maximumWeight = Mathf.Clamp(maximumWeight, 0f, 100f);
var center = new float[channels.Count];
var left = new float[channels.Count];
var right = new float[channels.Count];
var up = new float[channels.Count];
var down = new float[channels.Count];
// StandardEyeLookNames order: Up L/R, Down L/R, In L/R, Out L/R.
up[standardIndices[0]] = maximumWeight;
up[standardIndices[1]] = maximumWeight;
down[standardIndices[2]] = maximumWeight;
down[standardIndices[3]] = maximumWeight;
right[standardIndices[4]] = maximumWeight;
left[standardIndices[5]] = maximumWeight;
left[standardIndices[6]] = maximumWeight;
right[standardIndices[7]] = maximumWeight;
EnsureGrid();
samples[(int)GazeCalibrationPoint.Center].SetWeights(center, channels.Count);
samples[(int)GazeCalibrationPoint.CenterLeft].SetWeights(left, channels.Count);
samples[(int)GazeCalibrationPoint.CenterRight].SetWeights(right, channels.Count);
samples[(int)GazeCalibrationPoint.UpCenter].SetWeights(up, channels.Count);
samples[(int)GazeCalibrationPoint.DownCenter].SetWeights(down, channels.Count);
return GenerateCornersFromCardinals();
}
public bool CopySample(
GazeCalibrationPoint source,
GazeCalibrationPoint destination,
bool mirrorLeftRight)
{
if (!IsSampleCaptured(source))
return false;
var destinationIndex = (int)destination;
if (destinationIndex < 0 || destinationIndex >= 9)
return false;
var sourceWeights = GetSampleWeightsCopy(source);
if (!mirrorLeftRight)
{
samples[destinationIndex].SetWeights(sourceWeights, channels.Count);
return true;
}
var mirrored = new float[channels.Count];
for (var i = 0; i < mirrored.Length; i++)
{
var counterpart = FindMirroredChannelIndex(i);
mirrored[i] = sourceWeights[counterpart];
}
samples[destinationIndex].SetWeights(mirrored, channels.Count);
return true;
}
public void SetDirectionLimits(float left, float right, float down, float up)
{
minYaw = SanitizeNegativeDirectionLimit(left, DefaultMinYaw);
maxYaw = SanitizePositiveDirectionLimit(right, DefaultMaxYaw);
minPitch = SanitizeNegativeDirectionLimit(down, DefaultMinPitch);
maxPitch = SanitizePositiveDirectionLimit(up, DefaultMaxPitch);
}
public void SetSourceMesh(Mesh mesh)
{
sourceMesh = mesh;
}
public bool IsCompatibleWith(Mesh mesh)
{
return mesh != null && (sourceMesh == null || sourceMesh == mesh);
}
public bool SyncEyeLookChannels(SkinnedMeshRenderer renderer)
{
var mesh = renderer != null ? renderer.sharedMesh : null;
if (mesh == null)
return false;
var previousChannels = new List<GazeBlendShapeChannel>(channels);
var synchronized = new List<GazeBlendShapeChannel>();
var matchedEyeChannelCount = 0;
for (var standardIndex = 0; standardIndex < StandardEyeLookNames.Length; standardIndex++)
{
var standardName = StandardEyeLookNames[standardIndex];
var meshName = FindBlendShapeName(mesh, standardName);
if (string.IsNullOrEmpty(meshName))
continue;
var existing = FindChannel(previousChannels, meshName);
synchronized.Add(existing ?? new GazeBlendShapeChannel(meshName));
matchedEyeChannelCount++;
}
if (matchedEyeChannelCount != StandardEyeLookNames.Length)
return false;
// Preserve explicitly configured correctives when resyncing the eight eye channels.
for (var i = 0; i < previousChannels.Count; i++)
{
var channel = previousChannels[i];
if (channel != null && channel.Usage == GazeChannelUsage.Corrective
&& !ContainsChannel(synchronized, channel.BlendShapeName))
synchronized.Add(channel);
}
if (synchronized.Count == 0)
return false;
channels = synchronized;
EnsureGrid();
InvalidateSamples();
ResizeSamples();
UpdateChannelLayoutSignature();
sourceMesh = mesh;
return true;
}
public bool CaptureSample(GazeCalibrationPoint point, SkinnedMeshRenderer renderer)
{
if (renderer == null || renderer.sharedMesh == null || channels.Count == 0
|| !IsCompatibleWith(renderer.sharedMesh))
return false;
if (sourceMesh == null)
sourceMesh = renderer.sharedMesh;
var sampleIndex = (int)point;
if (sampleIndex < 0 || sampleIndex >= 9)
return false;
EnsureGrid();
var values = new float[channels.Count];
for (var i = 0; i < channels.Count; i++)
{
var channel = channels[i];
var index = channel != null
? FindBlendShapeIndex(renderer.sharedMesh, channel.BlendShapeName)
: -1;
if (index < 0)
return false;
values[i] = renderer.GetBlendShapeWeight(index);
}
samples[sampleIndex].SetWeights(values, channels.Count);
return true;
}
public void SetSample(GazeCalibrationPoint point, IReadOnlyList<float> weights)
{
EnsureGrid();
var sampleIndex = (int)point;
if (sampleIndex < 0 || sampleIndex >= samples.Count)
throw new ArgumentOutOfRangeException(nameof(point), point, "Calibration point must be in the 3x3 grid.");
if (weights == null || weights.Count != channels.Count)
throw new ArgumentException("A sample must contain exactly one weight per configured channel.", nameof(weights));
for (var channelIndex = 0; channelIndex < weights.Count; channelIndex++)
{
if (!IsFinite(weights[channelIndex]))
throw new ArgumentException("Calibration sample weights must be finite.", nameof(weights));
}
samples[sampleIndex].SetWeights(weights, channels.Count);
}
public float GetSampleWeight(GazeCalibrationPoint point, int channelIndex)
{
EnsureGrid();
var sampleIndex = (int)point;
return sampleIndex >= 0 && sampleIndex < samples.Count
&& channelIndex >= 0 && channelIndex < channels.Count
? samples[sampleIndex].GetWeight(channelIndex)
: 0f;
}
public void ReplaceChannels(IReadOnlyList<GazeBlendShapeChannel> replacements)
{
if (channels == null)
channels = new List<GazeBlendShapeChannel>();
channels.Clear();
if (replacements != null)
{
for (var i = 0; i < replacements.Count; i++)
{
var channel = replacements[i];
if (channel != null)
channels.Add(new GazeBlendShapeChannel(
channel.BlendShapeName,
channel.Usage,
channel.CorrectiveBlendMode));
}
}
EnsureGrid();
InvalidateSamples();
ResizeSamples();
UpdateChannelLayoutSignature();
}
public bool TryEvaluate(
GazeDirectionResult direction,
bool enableCorrectives,
float[] output)
{
if (output == null || output.Length < channels.Count || !IsFullyCalibrated)
return false;
EnsureGrid();
GazeDirectionSolver.Evaluate3x3(
samples,
channels.Count,
direction.NormalizedYaw,
direction.NormalizedPitch,
output);
for (var i = 0; i < channels.Count; i++)
{
if (!IsFinite(output[i]))
return false;
}
if (!enableCorrectives)
{
for (var i = 0; i < channels.Count; i++)
{
if (channels[i] != null && channels[i].Usage == GazeChannelUsage.Corrective)
output[i] = float.NaN;
}
}
return true;
}
public int FindBlendShapeIndex(Mesh mesh, int channelIndex)
{
return mesh != null && channelIndex >= 0 && channelIndex < channels.Count
? FindBlendShapeIndex(mesh, channels[channelIndex].BlendShapeName)
: -1;
}
public static int FindBlendShapeIndex(Mesh mesh, string name)
{
if (mesh == null || string.IsNullOrWhiteSpace(name))
return -1;
var exact = mesh.GetBlendShapeIndex(name);
if (exact >= 0)
return exact;
for (var i = 0; i < mesh.blendShapeCount; i++)
{
var meshName = mesh.GetBlendShapeName(i);
if (meshName.EndsWith(name, StringComparison.OrdinalIgnoreCase))
return i;
}
return -1;
}
private static string FindBlendShapeName(Mesh mesh, string standardName)
{
var index = FindBlendShapeIndex(mesh, standardName);
return index >= 0 ? mesh.GetBlendShapeName(index) : null;
}
private static GazeBlendShapeChannel FindChannel(
IReadOnlyList<GazeBlendShapeChannel> source,
string name)
{
for (var i = 0; i < source.Count; i++)
{
var channel = source[i];
if (channel != null && string.Equals(
channel.BlendShapeName,
name,
StringComparison.OrdinalIgnoreCase))
return channel;
}
return null;
}
private static bool ContainsChannel(
IReadOnlyList<GazeBlendShapeChannel> source,
string name)
{
return FindChannel(source, name) != null;
}
private void SetGeneratedCorner(
GazeCalibrationPoint point,
IReadOnlyList<float> vertical,
IReadOnlyList<float> horizontal,
IReadOnlyList<float> center)
{
var weights = new float[channels.Count];
for (var i = 0; i < weights.Length; i++)
{
var value = vertical[i] + horizontal[i] - center[i];
weights[i] = ClampGeneratedWeight(i, value);
}
samples[(int)point].SetWeights(weights, channels.Count);
}
private float ClampGeneratedWeight(int channelIndex, float value)
{
return channels[channelIndex] != null
&& channels[channelIndex].Usage == GazeChannelUsage.Corrective
? Mathf.Clamp(value, -100f, 100f)
: Mathf.Clamp(value, 0f, 100f);
}
private int FindChannelIndexByNormalizedSuffix(string standardName)
{
var normalizedSuffix = NormalizeChannelName(standardName);
for (var i = 0; i < channels.Count; i++)
{
var channel = channels[i];
var normalizedName = NormalizeChannelName(channel != null ? channel.BlendShapeName : null);
if (normalizedName.EndsWith(normalizedSuffix, StringComparison.Ordinal))
return i;
}
return -1;
}
private int FindMirroredChannelIndex(int channelIndex)
{
if (channelIndex < 0 || channelIndex >= channels.Count || channels[channelIndex] == null)
return channelIndex;
const string leftSuffix = "left";
const string rightSuffix = "right";
var normalizedName = NormalizeChannelName(channels[channelIndex].BlendShapeName);
string counterpartName;
if (normalizedName.EndsWith(leftSuffix, StringComparison.Ordinal))
{
counterpartName = normalizedName.Substring(0, normalizedName.Length - leftSuffix.Length)
+ rightSuffix;
}
else if (normalizedName.EndsWith(rightSuffix, StringComparison.Ordinal))
{
counterpartName = normalizedName.Substring(0, normalizedName.Length - rightSuffix.Length)
+ leftSuffix;
}
else
{
return channelIndex;
}
for (var i = 0; i < channels.Count; i++)
{
var channel = channels[i];
if (channel != null && string.Equals(
NormalizeChannelName(channel.BlendShapeName),
counterpartName,
StringComparison.Ordinal))
return i;
}
return channelIndex;
}
private static string NormalizeChannelName(string value)
{
if (string.IsNullOrEmpty(value))
return string.Empty;
var builder = new StringBuilder(value.Length);
for (var i = 0; i < value.Length; i++)
{
if (char.IsLetterOrDigit(value[i]))
builder.Append(char.ToLowerInvariant(value[i]));
}
return builder.ToString();
}
private void OnEnable()
{
if (channels == null)
channels = new List<GazeBlendShapeChannel>();
EnsureGrid();
ResizeSamples();
if (string.IsNullOrEmpty(channelLayoutSignature))
UpdateChannelLayoutSignature();
}
private void OnValidate()
{
minYaw = SanitizeNegativeDirectionLimit(minYaw, DefaultMinYaw);
maxYaw = SanitizePositiveDirectionLimit(maxYaw, DefaultMaxYaw);
minPitch = SanitizeNegativeDirectionLimit(minPitch, DefaultMinPitch);
maxPitch = SanitizePositiveDirectionLimit(maxPitch, DefaultMaxPitch);
if (channels == null)
channels = new List<GazeBlendShapeChannel>();
else
channels.RemoveAll(channel => channel == null);
EnsureGrid();
var currentSignature = BuildChannelLayoutSignature();
if (!string.IsNullOrEmpty(channelLayoutSignature)
&& !string.Equals(channelLayoutSignature, currentSignature, StringComparison.Ordinal))
InvalidateSamples();
ResizeSamples();
channelLayoutSignature = currentSignature;
}
private static float SanitizeNegativeDirectionLimit(float value, float fallback)
{
if (float.IsNaN(value) || float.IsInfinity(value))
value = fallback;
return Mathf.Clamp(value, -90f, -0.1f);
}
private static float SanitizePositiveDirectionLimit(float value, float fallback)
{
if (float.IsNaN(value) || float.IsInfinity(value))
value = fallback;
return Mathf.Clamp(value, 0.1f, 90f);
}
private static bool IsFinite(float value)
{
return !float.IsNaN(value) && !float.IsInfinity(value);
}
private void EnsureGrid()
{
if (samples == null)
samples = new List<GazeCalibrationSample>(9);
while (samples.Count < 9)
samples.Add(new GazeCalibrationSample((GazeCalibrationPoint)samples.Count));
if (samples.Count > 9)
samples.RemoveRange(9, samples.Count - 9);
for (var i = 0; i < 9; i++)
{
if (samples[i] == null)
samples[i] = new GazeCalibrationSample((GazeCalibrationPoint)i);
else
samples[i].SetPoint((GazeCalibrationPoint)i);
}
}
private void ResizeSamples()
{
for (var i = 0; i < samples.Count; i++)
samples[i].Resize(channels.Count);
}
private void InvalidateSamples()
{
for (var i = 0; i < samples.Count; i++)
samples[i]?.Invalidate();
}
private void UpdateChannelLayoutSignature()
{
channelLayoutSignature = BuildChannelLayoutSignature();
}
private string BuildChannelLayoutSignature()
{
var builder = new StringBuilder(channels.Count * 24);
for (var i = 0; i < channels.Count; i++)
{
var channel = channels[i];
builder.Append(channel != null ? channel.BlendShapeName : "<null>");
builder.Append('|');
builder.Append(channel != null ? (int)channel.Usage : -1);
builder.Append('|');
builder.Append(channel != null ? (int)channel.CorrectiveBlendMode : -1);
builder.Append(';');
}
return builder.ToString();
}
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: 27e03e57ecb32e245b2ab8d0a44ba805

View File

@ -0,0 +1,124 @@
using System.Collections.Generic;
using UnityEngine;
namespace Streamingle.Gaze
{
/// <summary>
/// Pure direction and 3x3 interpolation helpers. Kept independent of the
/// component so they can also be reused by a future real-time input source.
/// </summary>
public static class GazeDirectionSolver
{
private const float MinimumAngleRange = 0.001f;
public static GazeDirectionResult Calculate(
Quaternion referenceRotation,
Vector3 worldDirection,
float minYaw,
float maxYaw,
float minPitch,
float maxPitch)
{
if (worldDirection.sqrMagnitude <= Mathf.Epsilon)
worldDirection = referenceRotation * Vector3.forward;
else
worldDirection.Normalize();
var localDirection = Quaternion.Inverse(referenceRotation) * worldDirection;
if (localDirection.sqrMagnitude <= Mathf.Epsilon)
localDirection = Vector3.forward;
else
localDirection.Normalize();
var planarLength = Mathf.Sqrt(
localDirection.x * localDirection.x + localDirection.z * localDirection.z);
var rawYaw = Mathf.Atan2(localDirection.x, localDirection.z) * Mathf.Rad2Deg;
var rawPitch = Mathf.Atan2(localDirection.y, planarLength) * Mathf.Rad2Deg;
NormalizeRanges(ref minYaw, ref maxYaw, ref minPitch, ref maxPitch);
var clampedYaw = Mathf.Clamp(rawYaw, minYaw, maxYaw);
var clampedPitch = Mathf.Clamp(rawPitch, minPitch, maxPitch);
return new GazeDirectionResult(
worldDirection,
localDirection,
rawYaw,
rawPitch,
clampedYaw,
clampedPitch,
NormalizeAsymmetric(clampedYaw, minYaw, maxYaw),
NormalizeAsymmetric(clampedPitch, minPitch, maxPitch));
}
public static Vector3 DirectionFromAngles(float yaw, float pitch)
{
return Quaternion.Euler(-pitch, yaw, 0f) * Vector3.forward;
}
public static float NormalizeAsymmetric(float value, float negativeLimit, float positiveLimit)
{
if (value < 0f)
return Mathf.Clamp(value / Mathf.Max(MinimumAngleRange, -negativeLimit), -1f, 0f);
return Mathf.Clamp(value / Mathf.Max(MinimumAngleRange, positiveLimit), 0f, 1f);
}
public static void Evaluate3x3(
IReadOnlyList<GazeCalibrationSample> samples,
int channelCount,
float normalizedYaw,
float normalizedPitch,
float[] output)
{
if (output == null || output.Length < channelCount)
return;
normalizedYaw = Mathf.Clamp(normalizedYaw, -1f, 1f);
normalizedPitch = Mathf.Clamp(normalizedPitch, -1f, 1f);
var x0 = normalizedYaw < 0f ? 0 : 1;
var x1 = x0 + 1;
var y0 = normalizedPitch < 0f ? 0 : 1;
var y1 = y0 + 1;
var tx = normalizedYaw < 0f ? normalizedYaw + 1f : normalizedYaw;
var ty = normalizedPitch < 0f ? normalizedPitch + 1f : normalizedPitch;
var bottomLeft = GetSample(samples, y0 * 3 + x0);
var bottomRight = GetSample(samples, y0 * 3 + x1);
var topLeft = GetSample(samples, y1 * 3 + x0);
var topRight = GetSample(samples, y1 * 3 + x1);
for (var channel = 0; channel < channelCount; channel++)
{
var bottom = Mathf.Lerp(
bottomLeft != null ? bottomLeft.GetWeight(channel) : 0f,
bottomRight != null ? bottomRight.GetWeight(channel) : 0f,
tx);
var top = Mathf.Lerp(
topLeft != null ? topLeft.GetWeight(channel) : 0f,
topRight != null ? topRight.GetWeight(channel) : 0f,
tx);
output[channel] = Mathf.Lerp(bottom, top, ty);
}
}
private static GazeCalibrationSample GetSample(
IReadOnlyList<GazeCalibrationSample> samples,
int index)
{
return samples != null && index >= 0 && index < samples.Count ? samples[index] : null;
}
private static void NormalizeRanges(
ref float minYaw,
ref float maxYaw,
ref float minPitch,
ref float maxPitch)
{
minYaw = Mathf.Min(-MinimumAngleRange, minYaw);
maxYaw = Mathf.Max(MinimumAngleRange, maxYaw);
minPitch = Mathf.Min(-MinimumAngleRange, minPitch);
maxPitch = Mathf.Max(MinimumAngleRange, maxPitch);
}
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: db711509b9605ce4eb75578c5cc07215

203
Runtime/Gaze/GazeTypes.cs Normal file
View File

@ -0,0 +1,203 @@
using System;
using System.Collections.Generic;
using UnityEngine;
namespace Streamingle.Gaze
{
/// <summary>
/// Row-major layout of the calibration grid. Down/Up describe gaze pitch and
/// Left/Right describe gaze yaw from the character's point of view.
/// </summary>
public enum GazeCalibrationPoint
{
DownLeft = 0,
DownCenter = 1,
DownRight = 2,
CenterLeft = 3,
Center = 4,
CenterRight = 5,
UpLeft = 6,
UpCenter = 7,
UpRight = 8
}
public enum GazeChannelUsage
{
EyeLook = 0,
Corrective = 1
}
public enum GazeCorrectiveBlendMode
{
AdditiveFromCenter = 0,
Override = 1
}
public enum GazeCutResponse
{
Snap = 0,
Smooth = 1
}
[Serializable]
public sealed class GazeBlendShapeChannel
{
[SerializeField]
private string blendShapeName;
[SerializeField]
private GazeChannelUsage usage;
[SerializeField]
[Tooltip("Corrective channels normally add their direction-dependent delta from the Center sample to the live facial value.")]
private GazeCorrectiveBlendMode correctiveBlendMode;
public GazeBlendShapeChannel(
string blendShapeName,
GazeChannelUsage usage = GazeChannelUsage.EyeLook,
GazeCorrectiveBlendMode correctiveBlendMode = GazeCorrectiveBlendMode.AdditiveFromCenter)
{
this.blendShapeName = blendShapeName;
this.usage = usage;
this.correctiveBlendMode = correctiveBlendMode;
}
public string BlendShapeName
{
get => blendShapeName;
set => blendShapeName = value;
}
public GazeChannelUsage Usage
{
get => usage;
set => usage = value;
}
public GazeCorrectiveBlendMode CorrectiveBlendMode
{
get => correctiveBlendMode;
set => correctiveBlendMode = value;
}
}
[Serializable]
public sealed class GazeCalibrationSample
{
[SerializeField]
private GazeCalibrationPoint point;
[SerializeField]
private bool captured;
[SerializeField]
private float[] weights = Array.Empty<float>();
public GazeCalibrationSample(GazeCalibrationPoint point)
{
this.point = point;
}
public GazeCalibrationPoint Point => point;
public bool Captured => captured;
public IReadOnlyList<float> Weights => weights;
internal float GetWeight(int index)
{
return weights != null && index >= 0 && index < weights.Length ? weights[index] : 0f;
}
internal void SetWeights(IReadOnlyList<float> values, int channelCount)
{
var count = Mathf.Max(0, channelCount);
if (weights == null || weights.Length != count)
weights = new float[count];
for (var i = 0; i < count; i++)
weights[i] = values != null && i < values.Count ? values[i] : 0f;
captured = true;
}
internal void Resize(int channelCount)
{
var count = Mathf.Max(0, channelCount);
if (weights != null && weights.Length == count)
return;
var resized = new float[count];
if (weights != null)
Array.Copy(weights, resized, Mathf.Min(weights.Length, resized.Length));
weights = resized;
captured = false;
}
internal void Invalidate()
{
captured = false;
}
internal void SetPoint(GazeCalibrationPoint value)
{
point = value;
}
}
/// <summary>
/// Command passed from Timeline (or a future real-time source) to the final
/// LateUpdate blendshape writer.
/// </summary>
public struct GazeTimelineState
{
public Vector3 TargetPosition;
public bool HasTargetPosition;
public Vector3 TargetDirection;
public bool HasTargetDirection;
public Transform Target;
public bool UseMainCamera;
public float Influence;
public bool EnableCorrectives;
public bool OverrideCutResponse;
public GazeCutResponse CutResponse;
public float CutTransitionDuration;
public bool ResetSmoothing;
public bool Deterministic;
}
public readonly struct GazeDirectionResult
{
public GazeDirectionResult(
Vector3 worldDirection,
Vector3 localDirection,
float rawYaw,
float rawPitch,
float clampedYaw,
float clampedPitch,
float normalizedYaw,
float normalizedPitch)
{
WorldDirection = worldDirection;
LocalDirection = localDirection;
RawYaw = rawYaw;
RawPitch = rawPitch;
ClampedYaw = clampedYaw;
ClampedPitch = clampedPitch;
NormalizedYaw = normalizedYaw;
NormalizedPitch = normalizedPitch;
}
public Vector3 WorldDirection { get; }
public Vector3 LocalDirection { get; }
public float RawYaw { get; }
public float RawPitch { get; }
public float ClampedYaw { get; }
public float ClampedPitch { get; }
public float NormalizedYaw { get; }
public float NormalizedPitch { get; }
public bool WasClamped => !Mathf.Approximately(RawYaw, ClampedYaw)
|| !Mathf.Approximately(RawPitch, ClampedPitch);
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: ec857c34fe2ba9b45ae6f0fd6864a82f

View File

@ -0,0 +1,16 @@
{
"name": "Streamingle.Gaze.Runtime",
"rootNamespace": "Streamingle.Gaze",
"references": [
"Unity.Timeline"
],
"includePlatforms": [],
"excludePlatforms": [],
"allowUnsafeCode": false,
"overrideReferences": false,
"precompiledReferences": [],
"autoReferenced": true,
"defineConstraints": [],
"versionDefines": [],
"noEngineReferences": false
}

View File

@ -0,0 +1,7 @@
fileFormatVersion: 2
guid: 8b0cc41e29441174ab8738a308dd8d9c
AssemblyDefinitionImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

View File

@ -0,0 +1,8 @@
fileFormatVersion: 2
guid: c7b58b99b68445c4a96a4ad22d8a2bcb
folderAsset: yes
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

View File

@ -0,0 +1,23 @@
using UnityEngine;
using UnityEngine.Playables;
namespace Streamingle.Gaze
{
/// <summary>
/// Runtime data copied from a <see cref="BlendshapeGazeClip"/>.
/// </summary>
public sealed class BlendshapeGazeBehaviour : PlayableBehaviour
{
public Transform Target { get; internal set; }
public float Strength { get; internal set; }
public bool EnableCorrectives { get; internal set; }
public bool OverrideCutResponse { get; internal set; }
public GazeCutResponse CutResponse { get; internal set; }
public float CutTransitionDuration { get; internal set; }
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: 07c9f3da83d6a724b91c78d798091a03

View File

@ -0,0 +1,104 @@
using UnityEngine;
using UnityEngine.Playables;
using UnityEngine.Timeline;
namespace Streamingle.Gaze
{
/// <summary>
/// Describes a camera/target gaze request on a Timeline track.
/// The track mixer combines requests and submits the result to a
/// <see cref="BlendshapeGazeDriver"/>; clips never write blendshapes directly.
/// </summary>
public sealed class BlendshapeGazeClip : PlayableAsset, ITimelineClipAsset
{
[SerializeField]
[Tooltip("Optional world-space target. When empty, the Driver Target Camera is used before Camera.main.")]
private ExposedReference<Transform> target;
[SerializeField, Range(0f, 1f)]
[Tooltip("Maximum gaze influence contributed by this clip.")]
private float strength = 1f;
[SerializeField]
[Tooltip("Allow optional eyelid/corrective channels from the calibration profile.")]
private bool enableCorrectives;
[SerializeField]
[Tooltip("Override the bound driver's camera-cut response while this clip is dominant.")]
private bool overrideCutResponse;
[SerializeField]
private GazeCutResponse cutResponse;
[SerializeField, Min(0f)]
[Tooltip("Transition duration used by cut-response modes that smooth between cameras.")]
private float cutTransitionDuration = 0.1f;
public ExposedReference<Transform> Target
{
get => target;
set => target = value;
}
public float Strength
{
get => strength;
set => strength = SanitizeStrength(value);
}
public bool EnableCorrectives
{
get => enableCorrectives;
set => enableCorrectives = value;
}
public bool OverrideCutResponse
{
get => overrideCutResponse;
set => overrideCutResponse = value;
}
public GazeCutResponse CutResponse
{
get => cutResponse;
set => cutResponse = value;
}
public float CutTransitionDuration
{
get => cutTransitionDuration;
set => cutTransitionDuration = SanitizeDuration(value);
}
public ClipCaps clipCaps => ClipCaps.Blending;
public override Playable CreatePlayable(PlayableGraph graph, GameObject owner)
{
var playable = ScriptPlayable<BlendshapeGazeBehaviour>.Create(graph);
var behaviour = playable.GetBehaviour();
behaviour.Target = target.Resolve(graph.GetResolver());
behaviour.Strength = SanitizeStrength(strength);
behaviour.EnableCorrectives = enableCorrectives;
behaviour.OverrideCutResponse = overrideCutResponse;
behaviour.CutResponse = cutResponse;
behaviour.CutTransitionDuration = SanitizeDuration(cutTransitionDuration);
return playable;
}
private static float SanitizeStrength(float value)
{
return float.IsNaN(value) || float.IsInfinity(value)
? 0f
: Mathf.Clamp01(value);
}
private static float SanitizeDuration(float value)
{
return float.IsNaN(value) || float.IsInfinity(value)
? 0f
: Mathf.Max(0f, value);
}
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: 3df39f801d7538f46933258e013ceb96

View File

@ -0,0 +1,286 @@
using System.Collections.Generic;
using System.Threading;
using UnityEngine;
using UnityEngine.Playables;
#if UNITY_EDITOR
using UnityEditor;
#endif
namespace Streamingle.Gaze
{
/// <summary>
/// Blends Timeline clip requests into one command for the bound driver.
/// </summary>
public sealed class BlendshapeGazeMixerBehaviour : PlayableBehaviour
{
private const float MinimumWeight = 0.0001f;
private static int nextSourceId;
#if UNITY_EDITOR
private static readonly HashSet<BlendshapeGazeDriver> PendingEditorEvaluations =
new HashSet<BlendshapeGazeDriver>();
private static bool editorEvaluationScheduled;
#endif
private readonly int sourceId = Interlocked.Increment(ref nextSourceId);
private BlendshapeGazeDriver boundDriver;
private bool hasSubmittedState;
private bool hasProcessedFrame;
public override void PrepareFrame(Playable playable, FrameData info)
{
if (hasSubmittedState && boundDriver != null)
boundDriver.PrepareForTimelineEvaluation();
}
public override void ProcessFrame(Playable playable, FrameData info, object playerData)
{
var driver = playerData as BlendshapeGazeDriver;
if (driver == null)
{
ClearBoundDriver(ShouldEvaluateImmediately());
return;
}
if (boundDriver != driver)
{
ClearBoundDriver(ShouldEvaluateImmediately());
boundDriver = driver;
}
var outputWeight = Mathf.Max(0f, info.effectiveWeight);
var totalWeight = 0f;
var gazeOriginPosition = driver.GetGazeOriginPosition();
var weightedTargetDirection = Vector3.zero;
var hasCommonTarget = false;
var hasDifferentTargets = false;
var hasUnresolvedTarget = false;
Transform commonTarget = null;
var dominantWeight = float.NegativeInfinity;
BlendshapeGazeBehaviour dominantBehaviour = null;
Camera resolvedCamera = null;
var inputCount = playable.GetInputCount();
for (var i = 0; i < inputCount; i++)
{
var inputPlayable = (ScriptPlayable<BlendshapeGazeBehaviour>)playable.GetInput(i);
if (!inputPlayable.IsValid())
continue;
var behaviour = inputPlayable.GetBehaviour();
var inputWeight = Mathf.Max(0f, playable.GetInputWeight(i));
var contribution = inputWeight * outputWeight * Mathf.Clamp01(behaviour.Strength);
if (contribution <= MinimumWeight)
continue;
totalWeight += contribution;
var requestedTarget = behaviour.Target;
if (!hasCommonTarget)
{
commonTarget = requestedTarget;
hasCommonTarget = true;
}
else if (commonTarget != requestedTarget)
{
hasDifferentTargets = true;
}
var resolvedTarget = requestedTarget;
if (resolvedTarget == null)
{
if (resolvedCamera == null)
resolvedCamera = driver.TargetCamera != null
? driver.TargetCamera
: Camera.main;
if (resolvedCamera != null)
resolvedTarget = resolvedCamera.transform;
}
if (resolvedTarget != null)
{
var targetOffset = resolvedTarget.position - gazeOriginPosition;
if (targetOffset.sqrMagnitude > MinimumWeight * MinimumWeight)
{
weightedTargetDirection += targetOffset.normalized * contribution;
}
else
{
hasUnresolvedTarget = true;
}
}
else
{
hasUnresolvedTarget = true;
}
if (contribution > dominantWeight)
{
dominantWeight = contribution;
dominantBehaviour = behaviour;
}
}
var canDeferTargetResolution = hasCommonTarget && !hasDifferentTargets;
// A shared Transform (including a null target meaning Main Camera) is
// deliberately deferred to the driver. This lets LateUpdate observe the
// final Cinemachine camera pose. Distinct overlapping targets must be
// collapsed to a fixed blended direction during Timeline evaluation.
// Directions, rather than positions, prevent a distant target from
// outweighing a nearby target with the same Timeline contribution.
var cannotBlendTargets = !canDeferTargetResolution
&& (weightedTargetDirection.sqrMagnitude <= MinimumWeight * MinimumWeight
|| hasUnresolvedTarget);
if (totalWeight <= MinimumWeight || dominantBehaviour == null || cannotBlendTargets)
{
ClearSubmittedState(ShouldEvaluateImmediately());
hasProcessedFrame = true;
return;
}
var resetSmoothing = !hasProcessedFrame || info.seekOccurred || info.timeLooped;
var deterministic = !Application.isPlaying
|| info.evaluationType == FrameData.EvaluationType.Evaluate
|| info.seekOccurred
|| info.timeLooped;
var state = new GazeTimelineState
{
TargetDirection = canDeferTargetResolution
? Vector3.zero
: weightedTargetDirection.normalized,
HasTargetDirection = !canDeferTargetResolution,
Target = canDeferTargetResolution ? commonTarget : null,
UseMainCamera = canDeferTargetResolution && commonTarget == null,
Influence = Mathf.Clamp01(totalWeight),
EnableCorrectives = dominantBehaviour.EnableCorrectives,
OverrideCutResponse = dominantBehaviour.OverrideCutResponse,
CutResponse = dominantBehaviour.CutResponse,
CutTransitionDuration = dominantBehaviour.CutTransitionDuration,
ResetSmoothing = resetSmoothing,
Deterministic = deterministic
};
driver.SetTimelineState(sourceId, state);
hasSubmittedState = true;
hasProcessedFrame = true;
if (!Application.isPlaying)
{
// Evaluate after the whole Timeline graph has applied its animation
// outputs. This pass sees the animated GazeOrigin, final camera pose
// and facial blendshape baseline from this frame.
ScheduleEditorFinalEvaluation(driver);
}
}
public override void OnGraphStop(Playable playable)
{
ClearBoundDriver(ShouldEvaluateImmediately());
hasProcessedFrame = false;
}
public override void OnPlayableDestroy(Playable playable)
{
ClearBoundDriver(ShouldEvaluateImmediately());
hasProcessedFrame = false;
}
private void ClearSubmittedState(bool evaluateImmediately)
{
if (!hasSubmittedState)
return;
if (boundDriver != null)
{
boundDriver.ClearTimelineState(sourceId);
if (evaluateImmediately)
{
#if UNITY_EDITOR
ScheduleEditorFinalEvaluation(boundDriver);
#else
boundDriver.EvaluateNow(true);
#endif
}
}
hasSubmittedState = false;
}
private void ClearBoundDriver(bool evaluateImmediately)
{
ClearSubmittedState(evaluateImmediately);
boundDriver = null;
}
private static bool ShouldEvaluateImmediately()
{
return !Application.isPlaying;
}
#if UNITY_EDITOR
internal static void ScheduleEditorFinalEvaluation(BlendshapeGazeDriver driver)
{
if (driver == null)
return;
PendingEditorEvaluations.Add(driver);
if (editorEvaluationScheduled)
return;
editorEvaluationScheduled = true;
EditorApplication.delayCall += FlushPendingEditorEvaluations;
// Manual/paused Timeline evaluations do not guarantee an ExecuteAlways
// LateUpdate. Request one so the driver gets a post-animation final pass.
EditorApplication.QueuePlayerLoopUpdate();
}
internal static void FlushPendingEditorEvaluations()
{
EditorApplication.delayCall -= FlushPendingEditorEvaluations;
editorEvaluationScheduled = false;
if (PendingEditorEvaluations.Count == 0)
return;
var drivers = new BlendshapeGazeDriver[PendingEditorEvaluations.Count];
PendingEditorEvaluations.CopyTo(drivers);
PendingEditorEvaluations.Clear();
if (EditorApplication.isPlayingOrWillChangePlaymode)
return;
var evaluatedAny = false;
for (var index = 0; index < drivers.Length; index++)
{
if (drivers[index] != null && drivers[index].isActiveAndEnabled)
{
drivers[index].EvaluateNow(true);
evaluatedAny = true;
}
}
if (evaluatedAny)
SceneView.RepaintAll();
}
internal static void CancelEditorFinalEvaluation(BlendshapeGazeDriver driver)
{
if (object.ReferenceEquals(driver, null))
return;
PendingEditorEvaluations.Remove(driver);
if (PendingEditorEvaluations.Count > 0 || !editorEvaluationScheduled)
return;
EditorApplication.delayCall -= FlushPendingEditorEvaluations;
editorEvaluationScheduled = false;
}
#endif
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: 84259821b954cf84f853e4a88b17e80d

View File

@ -0,0 +1,78 @@
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.Playables;
using UnityEngine.Timeline;
namespace Streamingle.Gaze
{
[TrackColor(0.23f, 0.74f, 0.92f)]
[TrackBindingType(typeof(BlendshapeGazeDriver))]
[TrackClipType(typeof(BlendshapeGazeClip))]
public sealed class BlendshapeGazeTrack : TrackAsset
{
public override Playable CreateTrackMixer(PlayableGraph graph, GameObject go, int inputCount)
{
return ScriptPlayable<BlendshapeGazeMixerBehaviour>.Create(graph, inputCount);
}
#if UNITY_EDITOR
public override void GatherProperties(PlayableDirector director, IPropertyCollector collector)
{
var gazeDriver = director != null
? director.GetGenericBinding(this) as BlendshapeGazeDriver
: null;
var targetRenderer = gazeDriver != null ? gazeDriver.TargetRenderer : null;
var profile = gazeDriver != null ? gazeDriver.Profile : null;
var channels = profile != null ? profile.Channels : null;
var mesh = targetRenderer != null ? targetRenderer.sharedMesh : null;
if (targetRenderer != null && mesh != null && channels != null)
{
var channelNames = new HashSet<string>();
var previewBindings = new AnimationClip
{
name = "Blendshape Gaze Preview Bindings",
hideFlags = HideFlags.HideAndDontSave
};
try
{
for (var i = 0; i < channels.Count; i++)
{
var channel = channels[i];
if (object.ReferenceEquals(channel, null))
continue;
var blendShapeIndex = profile.FindBlendShapeIndex(mesh, i);
if (blendShapeIndex < 0)
continue;
// AddFromName expects a SerializedProperty path, while
// blendShape.* is a synthetic Animator curve binding.
// Register it through a clip so Timeline's AnimationMode
// resolves and restores the real renderer property.
var channelName = mesh.GetBlendShapeName(blendShapeIndex);
if (string.IsNullOrWhiteSpace(channelName) || !channelNames.Add(channelName))
continue;
previewBindings.SetCurve(
string.Empty,
typeof(SkinnedMeshRenderer),
"blendShape." + channelName,
AnimationCurve.Constant(0f, 1f, 0f));
}
if (channelNames.Count > 0)
collector.AddFromClip(targetRenderer.gameObject, previewBindings);
}
finally
{
Object.DestroyImmediate(previewBindings);
}
}
base.GatherProperties(director, collector);
}
#endif
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: 6c2beb3d591978e49bbc25526fa7fc51

View File

@ -0,0 +1,14 @@
{
"name": "Streamingle.Utilities.Runtime",
"rootNamespace": "Streamingle.Utils",
"references": [],
"includePlatforms": [],
"excludePlatforms": [],
"allowUnsafeCode": false,
"overrideReferences": false,
"precompiledReferences": [],
"autoReferenced": true,
"defineConstraints": [],
"versionDefines": [],
"noEngineReferences": false
}

View File

@ -0,0 +1,7 @@
fileFormatVersion: 2
guid: 3cb8039709b947446a09709edb79399e
AssemblyDefinitionImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

8
Runtime/Timeline.meta Normal file
View File

@ -0,0 +1,8 @@
fileFormatVersion: 2
guid: 7e3f7c38c8549684484fa5090615441f
folderAsset: yes
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

View File

@ -0,0 +1,8 @@
fileFormatVersion: 2
guid: 84ede7a9d3d8a8342a1a4958bbdc19d3
folderAsset: yes
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

View File

@ -0,0 +1,17 @@
{
"name": "Streamingle.Subtitles.Runtime",
"rootNamespace": "Streamingle.Subtitles",
"references": [
"Unity.TextMeshPro",
"Unity.Timeline"
],
"includePlatforms": [],
"excludePlatforms": [],
"allowUnsafeCode": false,
"overrideReferences": false,
"precompiledReferences": [],
"autoReferenced": true,
"defineConstraints": [],
"versionDefines": [],
"noEngineReferences": false
}

View File

@ -0,0 +1,7 @@
fileFormatVersion: 2
guid: 6d0f424a65eb1e748b892145abb71fc2
AssemblyDefinitionImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

View File

@ -0,0 +1,11 @@
using System;
using UnityEngine.Playables;
namespace Streamingle.Subtitles
{
[Serializable]
public sealed class SubtitleBehaviour : PlayableBehaviour
{
public string Text = string.Empty;
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: 6f48d9147e7499c4a9c572e2802ca17b

View File

@ -0,0 +1,27 @@
using UnityEngine;
using UnityEngine.Playables;
using UnityEngine.Timeline;
namespace Streamingle.Subtitles
{
public sealed class SubtitleClip : PlayableAsset, ITimelineClipAsset
{
[SerializeField, TextArea(1, 3)]
private string text = string.Empty;
public string Text
{
get => text;
set => text = value ?? string.Empty;
}
public ClipCaps clipCaps => ClipCaps.None;
public override Playable CreatePlayable(PlayableGraph graph, GameObject owner)
{
var playable = ScriptPlayable<SubtitleBehaviour>.Create(graph);
playable.GetBehaviour().Text = text ?? string.Empty;
return playable;
}
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: 42cbc317c93c78348ab3c28587916567

View File

@ -0,0 +1,81 @@
using TMPro;
using UnityEngine;
using UnityEngine.Playables;
namespace Streamingle.Subtitles
{
public sealed class SubtitleMixerBehaviour : PlayableBehaviour
{
private const float MinimumWeight = 0.0001f;
private TMP_Text trackBinding;
private string appliedText;
public override void ProcessFrame(Playable playable, FrameData info, object playerData)
{
SetBinding(playerData as TMP_Text);
if (trackBinding == null)
return;
var selectedText = string.Empty;
var greatestWeight = MinimumWeight;
var inputCount = playable.GetInputCount();
for (var index = 0; index < inputCount; index++)
{
var inputWeight = playable.GetInputWeight(index) * Mathf.Max(0f, info.effectiveWeight);
if (inputWeight <= greatestWeight)
continue;
var inputPlayable = (ScriptPlayable<SubtitleBehaviour>)playable.GetInput(index);
selectedText = inputPlayable.GetBehaviour().Text ?? string.Empty;
greatestWeight = inputWeight;
}
ApplyText(selectedText);
}
public override void OnGraphStop(Playable playable)
{
RestoreDefault();
}
public override void OnPlayableDestroy(Playable playable)
{
RestoreDefault();
}
private void SetBinding(TMP_Text target)
{
if (target == trackBinding)
return;
RestoreDefault();
trackBinding = target;
if (trackBinding == null)
return;
appliedText = null;
}
private void ApplyText(string value)
{
value ??= string.Empty;
if (appliedText == value && trackBinding.text == value)
return;
trackBinding.text = value;
appliedText = value;
}
private void RestoreDefault()
{
if (trackBinding == null)
return;
trackBinding.text = string.Empty;
trackBinding = null;
appliedText = null;
}
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: 66f86280161858943871bd56298c6153

View File

@ -0,0 +1,29 @@
using TMPro;
using UnityEngine;
using UnityEngine.Playables;
using UnityEngine.Timeline;
namespace Streamingle.Subtitles
{
[TrackColor(0.16f, 0.72f, 0.86f)]
[TrackBindingType(typeof(TMP_Text))]
[TrackClipType(typeof(SubtitleClip))]
public sealed class SubtitleTrack : TrackAsset
{
public override Playable CreateTrackMixer(PlayableGraph graph, GameObject go, int inputCount)
{
return ScriptPlayable<SubtitleMixerBehaviour>.Create(graph, inputCount);
}
#if UNITY_EDITOR
public override void GatherProperties(PlayableDirector director, IPropertyCollector collector)
{
var target = director != null ? director.GetGenericBinding(this) as TMP_Text : null;
if (target != null)
collector.AddFromName<TMP_Text>(target.gameObject, "m_text");
base.GatherProperties(director, collector);
}
#endif
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: 3577eec94a4b72246bf7e47a771787b2

8
Tests.meta Normal file
View File

@ -0,0 +1,8 @@
fileFormatVersion: 2
guid: b7847beb95e7c8a4dbb3766f333e0f2b
folderAsset: yes
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

8
Tests/Editor/Gaze.meta Normal file
View File

@ -0,0 +1,8 @@
fileFormatVersion: 2
guid: 152fec42be1e1c549ad14303896b3ecd
folderAsset: yes
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

View File

@ -0,0 +1,865 @@
using System.Collections.Generic;
using System.Reflection;
using NUnit.Framework;
using UnityEditor;
using UnityEngine;
using UnityEngine.Playables;
using UnityEngine.Timeline;
namespace Streamingle.Gaze.Tests
{
public sealed class BlendshapeGazeDriverTests
{
[Test]
public void Driver_OverridesBlendsAndRestoresExternalFacialWeight()
{
var root = new GameObject("GazeDriverTest");
var originObject = new GameObject("Origin");
var mesh = CreateBlendShapeMesh("eyeLookUpLeft");
var profile = ScriptableObject.CreateInstance<BlendshapeGazeProfile>();
try
{
originObject.transform.SetParent(root.transform, false);
var renderer = root.AddComponent<SkinnedMeshRenderer>();
renderer.sharedMesh = mesh;
renderer.SetBlendShapeWeight(0, 20f);
profile.ReplaceChannels(new[]
{
new GazeBlendShapeChannel("eyeLookUpLeft")
});
for (var index = 0; index < 9; index++)
profile.SetSample((GazeCalibrationPoint)index, new[] { 80f });
var driver = root.AddComponent<BlendshapeGazeDriver>();
driver.TargetRenderer = renderer;
driver.Profile = profile;
driver.GazeOrigin = originObject.transform;
driver.SetRuntimeTargetPosition(Vector3.forward, 0.5f);
driver.EvaluateNow(true);
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(50f).Within(0.001f));
// Simulate the facial Animator/receiver writing a new base before LateUpdate.
renderer.SetBlendShapeWeight(0, 40f);
Assert.That(driver.RebuildCache(), Is.True);
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(40f).Within(0.001f));
driver.EvaluateNow(true);
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(60f).Within(0.001f));
driver.ClearRuntimeTarget();
driver.EvaluateNow(true);
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(40f).Within(0.001f));
}
finally
{
UnityEngine.Object.DestroyImmediate(root);
UnityEngine.Object.DestroyImmediate(mesh);
UnityEngine.Object.DestroyImmediate(profile);
}
}
[Test]
public void Driver_IgnoresNonFiniteInfluenceWithoutWritingBlendshapes()
{
var root = new GameObject("GazeNonFiniteInfluenceTest");
var mesh = CreateBlendShapeMesh("eyeLookHorizontal");
var profile = ScriptableObject.CreateInstance<BlendshapeGazeProfile>();
try
{
var renderer = root.AddComponent<SkinnedMeshRenderer>();
renderer.sharedMesh = mesh;
renderer.SetBlendShapeWeight(0, 23f);
profile.ReplaceChannels(new[]
{
new GazeBlendShapeChannel("eyeLookHorizontal")
});
for (var index = 0; index < 9; index++)
profile.SetSample((GazeCalibrationPoint)index, new[] { 80f });
var driver = root.AddComponent<BlendshapeGazeDriver>();
driver.TargetRenderer = renderer;
driver.Profile = profile;
driver.SetRuntimeTargetPosition(Vector3.forward * 10f, float.NaN);
driver.EvaluateNow(true);
Assert.That(driver.HasActiveRequest, Is.False);
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(23f).Within(0.001f));
}
finally
{
Object.DestroyImmediate(root);
Object.DestroyImmediate(mesh);
Object.DestroyImmediate(profile);
}
}
[Test]
public void Driver_AddsCorrectiveDeltaWithoutErasingExistingBlink()
{
var root = new GameObject("GazeCorrectiveTest");
var originObject = new GameObject("Origin");
var mesh = CreateBlendShapeMesh("eyeLookUpLeft", "eyeBlinkLeft");
var profile = ScriptableObject.CreateInstance<BlendshapeGazeProfile>();
try
{
originObject.transform.SetParent(root.transform, false);
var renderer = root.AddComponent<SkinnedMeshRenderer>();
renderer.sharedMesh = mesh;
renderer.SetBlendShapeWeight(0, 10f);
renderer.SetBlendShapeWeight(1, 50f);
profile.ReplaceChannels(new[]
{
new GazeBlendShapeChannel("eyeLookUpLeft"),
new GazeBlendShapeChannel("eyeBlinkLeft", GazeChannelUsage.Corrective)
});
for (var index = 0; index < 9; index++)
profile.SetSample((GazeCalibrationPoint)index, new[] { 80f, 30f });
profile.SetSample(GazeCalibrationPoint.Center, new[] { 80f, 20f });
var driver = root.AddComponent<BlendshapeGazeDriver>();
driver.TargetRenderer = renderer;
driver.Profile = profile;
driver.GazeOrigin = originObject.transform;
var upTarget = GazeDirectionSolver.DirectionFromAngles(0f, profile.MaxPitch);
driver.SetRuntimeTargetPosition(upTarget, 1f, true);
driver.EvaluateNow(true);
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(80f).Within(0.001f));
Assert.That(renderer.GetBlendShapeWeight(1), Is.EqualTo(60f).Within(0.001f));
renderer.SetBlendShapeWeight(1, 70f);
driver.EvaluateNow(true);
Assert.That(renderer.GetBlendShapeWeight(1), Is.EqualTo(80f).Within(0.001f));
driver.ClearRuntimeTarget();
Assert.That(renderer.GetBlendShapeWeight(1), Is.EqualTo(70f).Within(0.001f));
}
finally
{
UnityEngine.Object.DestroyImmediate(root);
UnityEngine.Object.DestroyImmediate(mesh);
UnityEngine.Object.DestroyImmediate(profile);
}
}
[Test]
public void Driver_UsesAnimatedGazeOriginRotationBeforeHeadReferenceFallback()
{
var root = new GameObject("AnimatedGazeOriginTest");
var animatedHead = new GameObject("Head");
var originObject = new GameObject("Face_GazeOrigin");
var mesh = CreateBlendShapeMesh("eyeLookHorizontal");
var profile = ScriptableObject.CreateInstance<BlendshapeGazeProfile>();
try
{
animatedHead.transform.SetParent(root.transform, false);
originObject.transform.SetParent(animatedHead.transform, false);
var renderer = root.AddComponent<SkinnedMeshRenderer>();
renderer.sharedMesh = mesh;
renderer.SetBlendShapeWeight(0, 20f);
profile.ReplaceChannels(new[]
{
new GazeBlendShapeChannel("eyeLookHorizontal")
});
for (var index = 0; index < 9; index++)
{
var column = index % 3;
profile.SetSample(
(GazeCalibrationPoint)index,
new[] { column == 0 ? 0f : column == 1 ? 50f : 100f });
}
var driver = root.AddComponent<BlendshapeGazeDriver>();
driver.TargetRenderer = renderer;
driver.Profile = profile;
driver.GazeOrigin = originObject.transform;
// Deliberately keep this fallback fixed. Gaze Origin must win.
driver.HeadReference = root.transform;
driver.SetRuntimeTargetPosition(Vector3.forward * 10f, 1f);
driver.EvaluateNow(true);
Assert.That(driver.LastDirection.RawYaw, Is.EqualTo(0f).Within(0.01f));
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(50f).Within(0.001f));
animatedHead.transform.rotation = Quaternion.Euler(0f, 17.5f, 0f);
driver.EvaluateNow(true);
Assert.That(driver.LastDirection.RawYaw, Is.EqualTo(-17.5f).Within(0.01f));
Assert.That(driver.LastDirection.NormalizedYaw, Is.EqualTo(-0.5f).Within(0.001f));
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(25f).Within(0.001f));
}
finally
{
Object.DestroyImmediate(root);
Object.DestroyImmediate(mesh);
Object.DestroyImmediate(profile);
}
}
[Test]
public void Driver_ResolvesBlendedTargetDirectionFromLatestOriginPosition()
{
var root = new GameObject("LatestOriginDirectionTest");
var originObject = new GameObject("Face_GazeOrigin");
var mesh = CreateBlendShapeMesh("eyeLookHorizontal");
var profile = ScriptableObject.CreateInstance<BlendshapeGazeProfile>();
try
{
originObject.transform.SetParent(root.transform, false);
var renderer = root.AddComponent<SkinnedMeshRenderer>();
renderer.sharedMesh = mesh;
profile.ReplaceChannels(new[]
{
new GazeBlendShapeChannel("eyeLookHorizontal")
});
for (var index = 0; index < 9; index++)
{
var column = index % 3;
profile.SetSample(
(GazeCalibrationPoint)index,
new[] { column == 0 ? 0f : column == 1 ? 50f : 100f });
}
var driver = root.AddComponent<BlendshapeGazeDriver>();
driver.TargetRenderer = renderer;
driver.Profile = profile;
driver.GazeOrigin = originObject.transform;
var direction = GazeDirectionSolver.DirectionFromAngles(profile.MaxYaw, 0f);
driver.SetTimelineState(7119, new GazeTimelineState
{
TargetDirection = direction,
HasTargetDirection = true,
Influence = 1f
});
// Simulate an Animation Track moving the head after the gaze mixer
// submitted its blended direction.
originObject.transform.position = new Vector3(12f, 3f, -7f);
driver.EvaluateNow(true);
Assert.That(driver.LastDirection.RawYaw, Is.EqualTo(profile.MaxYaw).Within(0.01f));
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(100f).Within(0.001f));
Assert.That(
Vector3.Distance(
driver.LastTargetPosition,
originObject.transform.position + direction.normalized),
Is.LessThan(0.0001f));
}
finally
{
Object.DestroyImmediate(root);
Object.DestroyImmediate(mesh);
Object.DestroyImmediate(profile);
}
}
[Test]
public void Driver_UsesLowestSourceIdWhenTimelineInfluencesAreEqual()
{
var root = new GameObject("GazeTimelineTieBreakTest");
var mesh = CreateBlendShapeMesh("eyeLookHorizontal");
var profile = ScriptableObject.CreateInstance<BlendshapeGazeProfile>();
try
{
var renderer = root.AddComponent<SkinnedMeshRenderer>();
renderer.sharedMesh = mesh;
profile.ReplaceChannels(new[]
{
new GazeBlendShapeChannel("eyeLookHorizontal")
});
for (var index = 0; index < 9; index++)
{
var column = index % 3;
profile.SetSample(
(GazeCalibrationPoint)index,
new[] { column == 0 ? 0f : column == 1 ? 50f : 100f });
}
var driver = root.AddComponent<BlendshapeGazeDriver>();
driver.TargetRenderer = renderer;
driver.Profile = profile;
driver.SetTimelineState(20, new GazeTimelineState
{
TargetDirection = GazeDirectionSolver.DirectionFromAngles(profile.MaxYaw, 0f),
HasTargetDirection = true,
Influence = 1f
});
driver.SetTimelineState(10, new GazeTimelineState
{
TargetDirection = GazeDirectionSolver.DirectionFromAngles(profile.MinYaw, 0f),
HasTargetDirection = true,
Influence = 1f
});
driver.EvaluateNow(true);
Assert.That(driver.LastDirection.RawYaw, Is.EqualTo(profile.MinYaw).Within(0.01f));
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(0f).Within(0.001f));
}
finally
{
Object.DestroyImmediate(root);
Object.DestroyImmediate(mesh);
Object.DestroyImmediate(profile);
}
}
[Test]
public void EditorFinalPass_ComposesOverLatestFacialWrite()
{
var root = new GameObject("GazeEditorFinalPassTest");
var originObject = new GameObject("Face_GazeOrigin");
var mesh = CreateBlendShapeMesh("eyeLookUpLeft", "unrelatedBlink");
var profile = ScriptableObject.CreateInstance<BlendshapeGazeProfile>();
try
{
originObject.transform.SetParent(root.transform, false);
var renderer = root.AddComponent<SkinnedMeshRenderer>();
renderer.sharedMesh = mesh;
renderer.SetBlendShapeWeight(0, 20f);
renderer.SetBlendShapeWeight(1, 55f);
profile.ReplaceChannels(new[]
{
new GazeBlendShapeChannel("eyeLookUpLeft")
});
for (var index = 0; index < 9; index++)
profile.SetSample((GazeCalibrationPoint)index, new[] { 80f });
var driver = root.AddComponent<BlendshapeGazeDriver>();
driver.TargetRenderer = renderer;
driver.Profile = profile;
driver.GazeOrigin = originObject.transform;
const int sourceId = 8127;
driver.SetTimelineState(sourceId, new GazeTimelineState
{
TargetPosition = Vector3.forward * 10f,
HasTargetPosition = true,
Influence = 0.5f
});
// Timeline submits gaze first. The Animation/Facial output writes
// later in the same editor evaluation, before the queued final pass.
BlendshapeGazeMixerBehaviour.ScheduleEditorFinalEvaluation(driver);
root.transform.rotation = Quaternion.Euler(0f, 17.5f, 0f);
renderer.SetBlendShapeWeight(0, 40f);
renderer.SetBlendShapeWeight(1, 70f);
BlendshapeGazeMixerBehaviour.FlushPendingEditorEvaluations();
Assert.That(driver.LastDirection.RawYaw, Is.EqualTo(-17.5f).Within(0.01f));
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(60f).Within(0.001f));
Assert.That(renderer.GetBlendShapeWeight(1), Is.EqualTo(70f).Within(0.001f));
BlendshapeGazeMixerBehaviour.ScheduleEditorFinalEvaluation(driver);
renderer.SetBlendShapeWeight(0, 10f);
renderer.SetBlendShapeWeight(1, 35f);
BlendshapeGazeMixerBehaviour.FlushPendingEditorEvaluations();
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(45f).Within(0.001f));
Assert.That(renderer.GetBlendShapeWeight(1), Is.EqualTo(35f).Within(0.001f));
driver.ClearTimelineState(sourceId);
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(10f).Within(0.001f));
Assert.That(renderer.GetBlendShapeWeight(1), Is.EqualTo(35f).Within(0.001f));
}
finally
{
BlendshapeGazeMixerBehaviour.FlushPendingEditorEvaluations();
Object.DestroyImmediate(root);
Object.DestroyImmediate(mesh);
Object.DestroyImmediate(profile);
}
}
[Test]
public void EditorLateUpdate_ReappliesAfterAFacialWriteWithoutAnotherTimelineFrame()
{
var root = new GameObject("GazeEditorLateUpdateTest");
var originObject = new GameObject("Face_GazeOrigin");
var mesh = CreateBlendShapeMesh("eyeLookHorizontal");
var profile = ScriptableObject.CreateInstance<BlendshapeGazeProfile>();
try
{
originObject.transform.SetParent(root.transform, false);
var renderer = root.AddComponent<SkinnedMeshRenderer>();
renderer.sharedMesh = mesh;
renderer.SetBlendShapeWeight(0, 20f);
profile.ReplaceChannels(new[]
{
new GazeBlendShapeChannel("eyeLookHorizontal")
});
for (var index = 0; index < 9; index++)
profile.SetSample((GazeCalibrationPoint)index, new[] { 80f });
var driver = root.AddComponent<BlendshapeGazeDriver>();
driver.TargetRenderer = renderer;
driver.Profile = profile;
driver.GazeOrigin = originObject.transform;
const int sourceId = 9231;
driver.SetTimelineState(sourceId, new GazeTimelineState
{
TargetPosition = Vector3.forward * 10f,
HasTargetPosition = true,
Influence = 0.5f
});
driver.EvaluateNow(true);
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(50f).Within(0.001f));
// A later editor writer overwrites gaze without causing another
// Timeline ProcessFrame. ExecuteAlways LateUpdate must still win.
renderer.SetBlendShapeWeight(0, 10f);
originObject.transform.rotation = Quaternion.Euler(0f, 17.5f, 0f);
InvokePrivateLateUpdate(driver);
Assert.That(driver.LastDirection.RawYaw, Is.EqualTo(-17.5f).Within(0.01f));
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(45f).Within(0.001f));
driver.ClearTimelineState(sourceId);
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(10f).Within(0.001f));
}
finally
{
Object.DestroyImmediate(root);
Object.DestroyImmediate(mesh);
Object.DestroyImmediate(profile);
}
}
[Test]
public void TimelinePreparation_CapturesEqualFacialValueWithoutFeedback()
{
var root = new GameObject("GazeEditorBaselineCollisionTest");
var originObject = new GameObject("Face_GazeOrigin");
var mesh = CreateBlendShapeMesh("animatedEye", "unanimatedEye");
var profile = ScriptableObject.CreateInstance<BlendshapeGazeProfile>();
try
{
originObject.transform.SetParent(root.transform, false);
var renderer = root.AddComponent<SkinnedMeshRenderer>();
renderer.sharedMesh = mesh;
renderer.SetBlendShapeWeight(0, 80f);
renderer.SetBlendShapeWeight(1, 20f);
profile.ReplaceChannels(new[]
{
new GazeBlendShapeChannel("animatedEye"),
new GazeBlendShapeChannel("unanimatedEye")
});
for (var index = 0; index < 9; index++)
profile.SetSample((GazeCalibrationPoint)index, new[] { 0f, 80f });
var driver = root.AddComponent<BlendshapeGazeDriver>();
driver.TargetRenderer = renderer;
driver.Profile = profile;
driver.GazeOrigin = originObject.transform;
const int sourceId = 9232;
driver.SetTimelineState(sourceId, new GazeTimelineState
{
TargetPosition = Vector3.forward * 10f,
HasTargetPosition = true,
Influence = 0.5f
});
driver.EvaluateNow(true);
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(40f).Within(0.001f));
Assert.That(renderer.GetBlendShapeWeight(1), Is.EqualTo(50f).Within(0.001f));
driver.PrepareForTimelineEvaluation();
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(80f).Within(0.001f));
Assert.That(renderer.GetBlendShapeWeight(1), Is.EqualTo(20f).Within(0.001f));
// The next facial sample happens to equal the previous gaze result.
renderer.SetBlendShapeWeight(0, 40f);
for (var index = 0; index < 9; index++)
profile.SetSample((GazeCalibrationPoint)index, new[] { 100f, 80f });
driver.EvaluateNow(true);
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(70f).Within(0.001f));
Assert.That(renderer.GetBlendShapeWeight(1), Is.EqualTo(50f).Within(0.001f));
for (var iteration = 0; iteration < 5; iteration++)
{
driver.PrepareForTimelineEvaluation();
renderer.SetBlendShapeWeight(0, 40f);
driver.EvaluateNow(true);
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(70f).Within(0.001f));
Assert.That(renderer.GetBlendShapeWeight(1), Is.EqualTo(50f).Within(0.001f));
}
driver.ClearTimelineState(sourceId);
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(40f).Within(0.001f));
Assert.That(renderer.GetBlendShapeWeight(1), Is.EqualTo(20f).Within(0.001f));
}
finally
{
Object.DestroyImmediate(root);
Object.DestroyImmediate(mesh);
Object.DestroyImmediate(profile);
}
}
[Test]
public void DisablingDriver_CancelsQueuedEditorFinalEvaluation()
{
var root = new GameObject("GazeEditorCancellationTest");
var originObject = new GameObject("Face_GazeOrigin");
var mesh = CreateBlendShapeMesh("eyeLookHorizontal");
var profile = ScriptableObject.CreateInstance<BlendshapeGazeProfile>();
try
{
originObject.transform.SetParent(root.transform, false);
var renderer = root.AddComponent<SkinnedMeshRenderer>();
renderer.sharedMesh = mesh;
renderer.SetBlendShapeWeight(0, 23f);
profile.ReplaceChannels(new[]
{
new GazeBlendShapeChannel("eyeLookHorizontal")
});
for (var index = 0; index < 9; index++)
profile.SetSample((GazeCalibrationPoint)index, new[] { 80f });
var driver = root.AddComponent<BlendshapeGazeDriver>();
driver.TargetRenderer = renderer;
driver.Profile = profile;
driver.GazeOrigin = originObject.transform;
driver.SetTimelineState(9233, new GazeTimelineState
{
TargetPosition = Vector3.forward * 10f,
HasTargetPosition = true,
Influence = 1f
});
BlendshapeGazeMixerBehaviour.ScheduleEditorFinalEvaluation(driver);
driver.enabled = false;
BlendshapeGazeMixerBehaviour.FlushPendingEditorEvaluations();
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(23f).Within(0.001f));
Assert.That(driver.HasActiveRequest, Is.False);
}
finally
{
BlendshapeGazeMixerBehaviour.FlushPendingEditorEvaluations();
Object.DestroyImmediate(root);
Object.DestroyImmediate(mesh);
Object.DestroyImmediate(profile);
}
}
[Test]
public void GatherProperties_RegistersResolvedBlendShapeAsAnimationBinding()
{
var root = new GameObject("GazeGatherPropertiesTest");
var mesh = CreateBlendShapeMesh("Face.eyeLookUpLeft");
var profile = ScriptableObject.CreateInstance<BlendshapeGazeProfile>();
var track = ScriptableObject.CreateInstance<BlendshapeGazeTrack>();
try
{
var renderer = root.AddComponent<SkinnedMeshRenderer>();
renderer.sharedMesh = mesh;
renderer.SetBlendShapeWeight(0, 37f);
profile.ReplaceChannels(new[]
{
new GazeBlendShapeChannel("eyeLookUpLeft")
});
var driver = root.AddComponent<BlendshapeGazeDriver>();
driver.TargetRenderer = renderer;
driver.Profile = profile;
var director = root.AddComponent<PlayableDirector>();
director.SetGenericBinding(track, driver);
var collector = new RecordingPropertyCollector();
track.GatherProperties(director, collector);
Assert.That(collector.CurveProperties, Has.Count.EqualTo(1));
Assert.That(collector.CurveProperties[0].Root, Is.SameAs(root));
Assert.That(collector.CurveProperties[0].Binding.path, Is.Empty);
Assert.That(collector.CurveProperties[0].Binding.type, Is.EqualTo(typeof(SkinnedMeshRenderer)));
Assert.That(
collector.CurveProperties[0].Binding.propertyName,
Is.EqualTo("blendShape.Face.eyeLookUpLeft"));
// Exercise Timeline's real internal PropertyCollector. Unexpected
// DrivenPropertyManager errors fail the Unity test automatically.
// Use Timeline's own preview driver, while preserving any session
// already owned by an open Timeline/Animation window.
var previewDriver = GetTimelinePreviewDriver();
if (!AnimationMode.InAnimationMode())
{
var timelineCollector = CreateTimelinePropertyCollector();
AnimationMode.StartAnimationMode(previewDriver);
try
{
track.GatherProperties(director, timelineCollector);
renderer.SetBlendShapeWeight(0, 80f);
}
finally
{
AnimationMode.StopAnimationMode(previewDriver);
}
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(37f).Within(0.001f));
}
}
finally
{
Object.DestroyImmediate(root);
Object.DestroyImmediate(mesh);
Object.DestroyImmediate(profile);
Object.DestroyImmediate(track);
}
}
[Test]
public void ManualTimelineEvaluation_ComposesFacialAnimationAcrossRepeatedScrubs()
{
RunManualTimelineBaselineCollision(gazeTrackFirst: false);
RunManualTimelineBaselineCollision(gazeTrackFirst: true);
}
private static void RunManualTimelineBaselineCollision(bool gazeTrackFirst)
{
var root = new GameObject(gazeTrackFirst
? "GazeFirstManualTimelineTest"
: "AnimationFirstManualTimelineTest");
var originObject = new GameObject("Face_GazeOrigin");
var leftTarget = new GameObject("LeftTarget");
var rightTarget = new GameObject("RightTarget");
var mesh = CreateBlendShapeMesh("animatedEye");
var profile = ScriptableObject.CreateInstance<BlendshapeGazeProfile>();
var timeline = ScriptableObject.CreateInstance<TimelineAsset>();
var facialClip = new AnimationClip { name = "FacialBaseline" };
try
{
originObject.transform.SetParent(root.transform, false);
leftTarget.transform.SetParent(root.transform, false);
rightTarget.transform.SetParent(root.transform, false);
leftTarget.transform.position =
GazeDirectionSolver.DirectionFromAngles(profile.MinYaw, 0f) * 10f;
rightTarget.transform.position =
GazeDirectionSolver.DirectionFromAngles(profile.MaxYaw, 0f) * 10f;
var renderer = root.AddComponent<SkinnedMeshRenderer>();
renderer.sharedMesh = mesh;
renderer.SetBlendShapeWeight(0, 80f);
var animator = root.AddComponent<Animator>();
animator.cullingMode = AnimatorCullingMode.AlwaysAnimate;
profile.ReplaceChannels(new[]
{
new GazeBlendShapeChannel("animatedEye")
});
for (var index = 0; index < 9; index++)
{
var column = index % 3;
profile.SetSample(
(GazeCalibrationPoint)index,
new[] { column == 0 ? 0f : column == 1 ? 50f : 100f });
}
var driver = root.AddComponent<BlendshapeGazeDriver>();
driver.TargetRenderer = renderer;
driver.Profile = profile;
driver.GazeOrigin = originObject.transform;
AnimationTrack animationTrack;
BlendshapeGazeTrack gazeTrack;
if (gazeTrackFirst)
{
gazeTrack = timeline.CreateTrack<BlendshapeGazeTrack>(null, "Gaze");
animationTrack = timeline.CreateTrack<AnimationTrack>(null, "Facial");
}
else
{
animationTrack = timeline.CreateTrack<AnimationTrack>(null, "Facial");
gazeTrack = timeline.CreateTrack<BlendshapeGazeTrack>(null, "Gaze");
}
facialClip.SetCurve(
string.Empty,
typeof(SkinnedMeshRenderer),
"blendShape.animatedEye",
new AnimationCurve(
new Keyframe(0f, 80f),
new Keyframe(0.25f, 80f),
new Keyframe(0.75f, 40f),
new Keyframe(1f, 40f)));
var facialTimelineClip = animationTrack.CreateClip(facialClip);
facialTimelineClip.start = 0d;
facialTimelineClip.duration = 1d;
var leftClip = gazeTrack.CreateClip<BlendshapeGazeClip>();
leftClip.start = 0d;
leftClip.duration = 0.5d;
var leftAsset = (BlendshapeGazeClip)leftClip.asset;
leftAsset.Target = new ExposedReference<Transform>
{
defaultValue = leftTarget.transform
};
leftAsset.Strength = 0.5f;
var rightClip = gazeTrack.CreateClip<BlendshapeGazeClip>();
rightClip.start = 0.5d;
rightClip.duration = 0.5d;
var rightAsset = (BlendshapeGazeClip)rightClip.asset;
rightAsset.Target = new ExposedReference<Transform>
{
defaultValue = rightTarget.transform
};
rightAsset.Strength = 0.5f;
var director = root.AddComponent<PlayableDirector>();
director.timeUpdateMode = DirectorUpdateMode.Manual;
director.playableAsset = timeline;
director.SetGenericBinding(animationTrack, animator);
director.SetGenericBinding(gazeTrack, driver);
director.time = 0.25d;
director.Evaluate();
BlendshapeGazeMixerBehaviour.FlushPendingEditorEvaluations();
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(40f).Within(0.01f));
director.time = 0.75d;
director.Evaluate();
BlendshapeGazeMixerBehaviour.FlushPendingEditorEvaluations();
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(70f).Within(0.01f));
for (var iteration = 0; iteration < 5; iteration++)
{
director.Evaluate();
BlendshapeGazeMixerBehaviour.FlushPendingEditorEvaluations();
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(70f).Within(0.01f));
}
director.time = 0.25d;
director.Evaluate();
BlendshapeGazeMixerBehaviour.FlushPendingEditorEvaluations();
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(40f).Within(0.01f));
}
finally
{
BlendshapeGazeMixerBehaviour.FlushPendingEditorEvaluations();
Object.DestroyImmediate(root);
Object.DestroyImmediate(mesh);
Object.DestroyImmediate(profile);
Object.DestroyImmediate(facialClip);
Object.DestroyImmediate(timeline);
}
}
private static void InvokePrivateLateUpdate(BlendshapeGazeDriver driver)
{
var method = typeof(BlendshapeGazeDriver).GetMethod(
"LateUpdate",
BindingFlags.Instance | BindingFlags.NonPublic);
Assert.That(method, Is.Not.Null);
method.Invoke(driver, null);
}
private static IPropertyCollector CreateTimelinePropertyCollector()
{
foreach (var assembly in System.AppDomain.CurrentDomain.GetAssemblies())
{
var type = assembly.GetType("UnityEditor.Timeline.PropertyCollector");
if (type != null)
return (IPropertyCollector)System.Activator.CreateInstance(type, true);
}
Assert.Fail("Unity Timeline PropertyCollector type was not loaded.");
return null;
}
private static AnimationModeDriver GetTimelinePreviewDriver()
{
foreach (var assembly in System.AppDomain.CurrentDomain.GetAssemblies())
{
var type = assembly.GetType("UnityEditor.Timeline.WindowState");
var property = type?.GetProperty(
"previewDriver",
BindingFlags.Static | BindingFlags.Public | BindingFlags.NonPublic);
if (property?.GetValue(null) is AnimationModeDriver previewDriver)
return previewDriver;
}
Assert.Fail("Unity Timeline preview driver was not loaded.");
return null;
}
private sealed class RecordingPropertyCollector : IPropertyCollector
{
internal readonly List<CurveProperty> CurveProperties =
new List<CurveProperty>();
public void PushActiveGameObject(GameObject gameObject) { }
public void PopActiveGameObject() { }
public void AddFromClip(AnimationClip clip) { }
public void AddFromClips(IEnumerable<AnimationClip> clips) { }
public void AddFromName<T>(string name) where T : Component { }
public void AddFromName(string name) { }
public void AddFromClip(GameObject obj, AnimationClip clip)
{
var bindings = AnimationUtility.GetCurveBindings(clip);
for (var index = 0; index < bindings.Length; index++)
CurveProperties.Add(new CurveProperty(obj, bindings[index]));
}
public void AddFromClips(GameObject obj, IEnumerable<AnimationClip> clips) { }
public void AddFromName<T>(GameObject obj, string name) where T : Component { }
public void AddFromName(GameObject obj, string name) { }
public void AddFromName(Component component, string name) { }
public void AddFromComponent(GameObject obj, Component component) { }
public void AddObjectProperties(Object obj, AnimationClip clip) { }
}
private readonly struct CurveProperty
{
internal CurveProperty(GameObject root, EditorCurveBinding binding)
{
Root = root;
Binding = binding;
}
internal GameObject Root { get; }
internal EditorCurveBinding Binding { get; }
}
private static Mesh CreateBlendShapeMesh(params string[] blendShapeNames)
{
var mesh = new Mesh { name = "GazeDriverTestMesh" };
mesh.vertices = new[]
{
new Vector3(-0.1f, 0f, 0f),
new Vector3(0.1f, 0f, 0f),
new Vector3(0f, 0.1f, 0f)
};
mesh.triangles = new[] { 0, 1, 2 };
for (var index = 0; index < blendShapeNames.Length; index++)
{
mesh.AddBlendShapeFrame(
blendShapeNames[index],
100f,
new[] { Vector3.up * 0.01f, Vector3.up * 0.01f, Vector3.up * 0.01f },
new Vector3[3],
new Vector3[3]);
}
return mesh;
}
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: 37c10ba76e48457438d8da1d6673d1f8

View File

@ -0,0 +1,224 @@
using NUnit.Framework;
using Streamingle.Gaze.Editor;
using UnityEngine;
namespace Streamingle.Gaze.Tests
{
public sealed class GazeCalibrationPreviewSessionTests
{
[Test]
public void Session_UsesRuntimeBlendSemantics_BlocksDriverAndRestoresWeights()
{
var root = new GameObject("CalibrationPreviewSessionTest");
var mesh = CreateMesh();
var profile = ScriptableObject.CreateInstance<BlendshapeGazeProfile>();
var session = new GazeCalibrationPreviewSession();
try
{
var renderer = root.AddComponent<SkinnedMeshRenderer>();
renderer.sharedMesh = mesh;
renderer.SetBlendShapeWeight(0, 25f);
renderer.SetBlendShapeWeight(1, 30f);
renderer.SetBlendShapeWeight(2, 45f);
renderer.SetBlendShapeWeight(3, 55f);
profile.ReplaceChannels(new[]
{
new GazeBlendShapeChannel("eyeLookUpLeft"),
new GazeBlendShapeChannel(
"eyeCorrectiveAdd",
GazeChannelUsage.Corrective,
GazeCorrectiveBlendMode.AdditiveFromCenter),
new GazeBlendShapeChannel(
"eyeCorrectiveOverride",
GazeChannelUsage.Corrective,
GazeCorrectiveBlendMode.Override)
});
for (var index = 0; index < 9; index++)
{
profile.SetSample(
(GazeCalibrationPoint)index,
new[]
{
index * 10f,
10f + index * 2f,
100f - index * 5f
});
}
var driver = root.AddComponent<BlendshapeGazeDriver>();
driver.TargetRenderer = renderer;
driver.Profile = profile;
var gazeOrigin = new GameObject("GazeOrigin").transform;
gazeOrigin.SetParent(root.transform, false);
driver.GazeOrigin = gazeOrigin;
Assert.That(session.Start(driver, out var error), Is.True, error);
Assert.That(session.GetOriginalWeightsCopy(), Is.EqualTo(new[] { 25f, 30f, 45f }));
session.PreviewSample(GazeCalibrationPoint.UpRight);
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(80f).Within(0.001f));
Assert.That(renderer.GetBlendShapeWeight(1), Is.EqualTo(38f).Within(0.001f));
Assert.That(renderer.GetBlendShapeWeight(2), Is.EqualTo(60f).Within(0.001f));
Assert.That(renderer.GetBlendShapeWeight(3), Is.EqualTo(55f).Within(0.001f));
session.PreviewWeights(new[] { 67f, 23f, 72f }, GazeCalibrationPoint.CenterRight);
Assert.That(session.IsFreePreview, Is.False);
Assert.That(session.PreviewCoordinates, Is.EqualTo(Vector2.right));
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(67f).Within(0.001f));
Assert.That(renderer.GetBlendShapeWeight(1), Is.EqualTo(35f).Within(0.001f));
Assert.That(renderer.GetBlendShapeWeight(2), Is.EqualTo(72f).Within(0.001f));
// The four surrounding samples are deliberately distinct. At
// (0.5, -0.5), indices 1/2/4/5 contribute equally.
session.PreviewNormalized(new Vector2(0.5f, -0.5f));
Assert.That(session.IsFreePreview, Is.True);
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(30f).Within(0.001f));
Assert.That(renderer.GetBlendShapeWeight(1), Is.EqualTo(28f).Within(0.001f));
Assert.That(renderer.GetBlendShapeWeight(2), Is.EqualTo(85f).Within(0.001f));
driver.SetTimelineState(1234, new GazeTimelineState
{
TargetPosition = new Vector3(1f, 0f, 1f),
HasTargetPosition = true,
Influence = 1f,
EnableCorrectives = true
});
driver.EvaluateNow(true);
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(30f).Within(0.001f));
Assert.That(renderer.GetBlendShapeWeight(1), Is.EqualTo(28f).Within(0.001f));
Assert.That(renderer.GetBlendShapeWeight(2), Is.EqualTo(85f).Within(0.001f));
session.Stop();
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(25f).Within(0.001f));
Assert.That(renderer.GetBlendShapeWeight(1), Is.EqualTo(30f).Within(0.001f));
Assert.That(renderer.GetBlendShapeWeight(2), Is.EqualTo(45f).Within(0.001f));
Assert.That(renderer.GetBlendShapeWeight(3), Is.EqualTo(55f).Within(0.001f));
// Releasing the session also releases the driver override. A yaw
// beyond the profile limit resolves to CenterRight (sample 5).
driver.EvaluateNow(true);
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(50f).Within(0.001f));
Assert.That(renderer.GetBlendShapeWeight(1), Is.EqualTo(32f).Within(0.001f));
Assert.That(renderer.GetBlendShapeWeight(2), Is.EqualTo(75f).Within(0.001f));
}
finally
{
session.Dispose();
Object.DestroyImmediate(root);
Object.DestroyImmediate(mesh);
Object.DestroyImmediate(profile);
}
}
[Test]
public void Session_DoesNotRestoreIntoAnIndexWhoseResolvedNameChanged()
{
var root = new GameObject("CalibrationPreviewResolvedNameTest");
var mesh = CreateMesh();
var profile = ScriptableObject.CreateInstance<BlendshapeGazeProfile>();
var session = new GazeCalibrationPreviewSession();
try
{
var renderer = root.AddComponent<SkinnedMeshRenderer>();
renderer.sharedMesh = mesh;
renderer.SetBlendShapeWeight(0, 25f);
profile.ReplaceChannels(new[] { new GazeBlendShapeChannel("eyeLookUpLeft") });
for (var index = 0; index < 9; index++)
profile.SetSample((GazeCalibrationPoint)index, new[] { 80f });
var driver = root.AddComponent<BlendshapeGazeDriver>();
driver.TargetRenderer = renderer;
driver.Profile = profile;
Assert.That(session.Start(driver, out var error), Is.True, error);
session.PreviewSample(GazeCalibrationPoint.Center);
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(80f).Within(0.001f));
mesh.ClearBlendShapes();
AddBlendShape(mesh, "replacementShape");
renderer.SetBlendShapeWeight(0, 66f);
session.Stop();
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(66f).Within(0.001f));
}
finally
{
session.Dispose();
Object.DestroyImmediate(root);
Object.DestroyImmediate(mesh);
Object.DestroyImmediate(profile);
}
}
[Test]
public void Session_AdditiveDraftUsesOriginalWeightUntilCenterIsCaptured()
{
var root = new GameObject("CalibrationPreviewDraftCorrectiveTest");
var mesh = CreateMesh();
var profile = ScriptableObject.CreateInstance<BlendshapeGazeProfile>();
var session = new GazeCalibrationPreviewSession();
try
{
var renderer = root.AddComponent<SkinnedMeshRenderer>();
renderer.sharedMesh = mesh;
renderer.SetBlendShapeWeight(1, 30f);
profile.ReplaceChannels(new[]
{
new GazeBlendShapeChannel(
"eyeCorrectiveAdd",
GazeChannelUsage.Corrective,
GazeCorrectiveBlendMode.AdditiveFromCenter)
});
var driver = root.AddComponent<BlendshapeGazeDriver>();
driver.TargetRenderer = renderer;
driver.Profile = profile;
Assert.That(session.Start(driver, out var error), Is.True, error);
session.PreviewWeights(new[] { 42f }, GazeCalibrationPoint.UpRight);
Assert.That(renderer.GetBlendShapeWeight(1), Is.EqualTo(42f).Within(0.001f));
session.Stop();
Assert.That(renderer.GetBlendShapeWeight(1), Is.EqualTo(30f).Within(0.001f));
}
finally
{
session.Dispose();
Object.DestroyImmediate(root);
Object.DestroyImmediate(mesh);
Object.DestroyImmediate(profile);
}
}
private static Mesh CreateMesh()
{
var mesh = new Mesh { name = "CalibrationPreviewSessionMesh" };
mesh.vertices = new[]
{
new Vector3(-0.1f, 0f, 0f),
new Vector3(0.1f, 0f, 0f),
new Vector3(0f, 0.1f, 0f)
};
mesh.triangles = new[] { 0, 1, 2 };
AddBlendShape(mesh, "Face.eyeLookUpLeft");
AddBlendShape(mesh, "Face.eyeCorrectiveAdd");
AddBlendShape(mesh, "Face.eyeCorrectiveOverride");
AddBlendShape(mesh, "unrelatedFaceShape");
return mesh;
}
private static void AddBlendShape(Mesh mesh, string name)
{
mesh.AddBlendShapeFrame(
name,
100f,
new[] { Vector3.up * 0.01f, Vector3.up * 0.01f, Vector3.up * 0.01f },
new Vector3[3],
new Vector3[3]);
}
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: 7ce304fdd09812a49baacc0f0d5c2cba

View File

@ -0,0 +1,250 @@
using System.Reflection;
using NUnit.Framework;
using Streamingle.Gaze.Editor;
using UnityEngine;
namespace Streamingle.Gaze.Tests
{
public sealed class GazeCalibrationWindowTests
{
[Test]
public void AutoCreateOrigin_ParentsFaceGazeOriginUnderRendererHeadBone()
{
var root = new GameObject("GazeOriginParentTest");
var head = new GameObject("Bip001 Head");
var mesh = CreateMesh();
var profile = ScriptableObject.CreateInstance<BlendshapeGazeProfile>();
var window = ScriptableObject.CreateInstance<GazeCalibrationWindow>();
try
{
head.transform.SetParent(root.transform, false);
var renderer = root.AddComponent<SkinnedMeshRenderer>();
renderer.sharedMesh = mesh;
renderer.bones = new[] { head.transform };
profile.ReplaceChannels(new[]
{
new GazeBlendShapeChannel("eyeLookUpLeft")
});
var driver = root.AddComponent<BlendshapeGazeDriver>();
driver.TargetRenderer = renderer;
driver.Profile = profile;
Invoke(window, "SetDriver", driver);
Invoke(window, "AutoCreateOrRecalculateOrigin");
Assert.That(driver.GazeOrigin, Is.Not.Null);
Assert.That(driver.GazeOrigin.name, Is.EqualTo("Face_GazeOrigin"));
Assert.That(driver.GazeOrigin.parent, Is.EqualTo(head.transform));
}
finally
{
if (window != null)
Object.DestroyImmediate(window);
Object.DestroyImmediate(root);
Object.DestroyImmediate(mesh);
Object.DestroyImmediate(profile);
}
}
[Test]
public void AutoCreateOrigin_ReusesExistingHeadChildWhenDriverReferenceIsMissing()
{
var root = new GameObject("GazeOriginReuseTest");
var head = new GameObject("Bip001 Head");
var existingOrigin = new GameObject("Face_GazeOrigin");
var mesh = CreateMesh();
var profile = ScriptableObject.CreateInstance<BlendshapeGazeProfile>();
var window = ScriptableObject.CreateInstance<GazeCalibrationWindow>();
try
{
head.transform.SetParent(root.transform, false);
existingOrigin.transform.SetParent(head.transform, false);
var renderer = root.AddComponent<SkinnedMeshRenderer>();
renderer.sharedMesh = mesh;
renderer.bones = new[] { head.transform };
profile.ReplaceChannels(new[]
{
new GazeBlendShapeChannel("eyeLookUpLeft")
});
var driver = root.AddComponent<BlendshapeGazeDriver>();
driver.TargetRenderer = renderer;
driver.Profile = profile;
Invoke(window, "SetDriver", driver);
Invoke(window, "AutoCreateOrRecalculateOrigin");
Assert.That(driver.GazeOrigin, Is.SameAs(existingOrigin.transform));
Assert.That(head.transform.childCount, Is.EqualTo(1));
}
finally
{
if (window != null)
Object.DestroyImmediate(window);
Object.DestroyImmediate(root);
Object.DestroyImmediate(mesh);
Object.DestroyImmediate(profile);
}
}
[Test]
public void SelectingMissingPointsDoesNotCaptureAndCancelRevertsProfile()
{
var root = new GameObject("GazeCalibrationWindowTest");
var mesh = CreateMesh();
var profile = ScriptableObject.CreateInstance<BlendshapeGazeProfile>();
var window = ScriptableObject.CreateInstance<GazeCalibrationWindow>();
try
{
var renderer = root.AddComponent<SkinnedMeshRenderer>();
renderer.sharedMesh = mesh;
renderer.SetBlendShapeWeight(0, 12f);
profile.ReplaceChannels(new[]
{
new GazeBlendShapeChannel("eyeLookUpLeft")
});
var origin = new GameObject("GazeOrigin").transform;
origin.SetParent(root.transform, false);
var driver = root.AddComponent<BlendshapeGazeDriver>();
driver.TargetRenderer = renderer;
driver.Profile = profile;
driver.GazeOrigin = origin;
Invoke(window, "SetDriver", driver);
Invoke(window, "StartCalibration");
Invoke(window, "SelectPoint", GazeCalibrationPoint.CenterLeft);
Assert.That(profile.IsSampleCaptured(GazeCalibrationPoint.Center), Is.False);
Assert.That(profile.IsSampleCaptured(GazeCalibrationPoint.CenterLeft), Is.False);
// Guided calibration edits an unsaved working copy. Cancel must
// discard it without ever mutating the original profile asset.
var session = GetPrivateField<GazeCalibrationPreviewSession>(window, "previewSession");
Assert.That(session, Is.Not.Null);
Assert.That(session.Profile, Is.Not.SameAs(profile));
UnityEditor.Undo.RecordObject(session.Profile, "Test Gaze Calibration Edit");
session.Profile.SetSample(GazeCalibrationPoint.CenterLeft, new[] { 77f });
renderer.SetBlendShapeWeight(0, 77f);
Assert.That(profile.IsSampleCaptured(GazeCalibrationPoint.CenterLeft), Is.False);
// A mapping/play-mode/reload guard may stop the preview before the
// window repaints. The stop callback must cancel immediately.
session.Stop();
Invoke(window, "FinishCalibration", false, false);
Assert.That(profile.IsSampleCaptured(GazeCalibrationPoint.CenterLeft), Is.False);
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(12f).Within(0.001f));
}
finally
{
if (window != null)
{
Invoke(window, "FinishCalibration", false, false);
Object.DestroyImmediate(window);
}
Object.DestroyImmediate(root);
Object.DestroyImmediate(mesh);
Object.DestroyImmediate(profile);
}
}
[Test]
public void ApplyCalibration_CommitsWorkingCopyAndRestoresRendererPose()
{
var root = new GameObject("GazeCalibrationApplyTest");
var mesh = CreateMesh();
var profile = ScriptableObject.CreateInstance<BlendshapeGazeProfile>();
var window = ScriptableObject.CreateInstance<GazeCalibrationWindow>();
try
{
var renderer = root.AddComponent<SkinnedMeshRenderer>();
renderer.sharedMesh = mesh;
renderer.SetBlendShapeWeight(0, 14f);
profile.ReplaceChannels(new[]
{
new GazeBlendShapeChannel("eyeLookUpLeft")
});
var origin = new GameObject("GazeOrigin").transform;
origin.SetParent(root.transform, false);
var driver = root.AddComponent<BlendshapeGazeDriver>();
driver.TargetRenderer = renderer;
driver.Profile = profile;
driver.GazeOrigin = origin;
Invoke(window, "SetDriver", driver);
Invoke(window, "StartCalibration");
var session = GetPrivateField<GazeCalibrationPreviewSession>(window, "previewSession");
Assert.That(session, Is.Not.Null);
Assert.That(session.Profile, Is.Not.SameAs(profile));
UnityEditor.Undo.RecordObject(session.Profile, "Test Apply Gaze Calibration");
session.Profile.SetSample(GazeCalibrationPoint.Center, new[] { 66f });
session.PreviewSample(GazeCalibrationPoint.Center);
Assert.That(profile.IsSampleCaptured(GazeCalibrationPoint.Center), Is.False);
Invoke(window, "FinishCalibration", true, false);
Assert.That(profile.IsSampleCaptured(GazeCalibrationPoint.Center), Is.True);
Assert.That(
profile.GetSampleWeight(GazeCalibrationPoint.Center, 0),
Is.EqualTo(66f).Within(0.001f));
Assert.That(renderer.GetBlendShapeWeight(0), Is.EqualTo(14f).Within(0.001f));
}
finally
{
if (window != null)
{
Invoke(window, "FinishCalibration", false, false);
Object.DestroyImmediate(window);
}
Object.DestroyImmediate(root);
Object.DestroyImmediate(mesh);
Object.DestroyImmediate(profile);
}
}
private static void Invoke(object target, string methodName, params object[] arguments)
{
var method = target.GetType().GetMethod(
methodName,
BindingFlags.Instance | BindingFlags.NonPublic);
Assert.That(method, Is.Not.Null, $"Expected private method '{methodName}'.");
method.Invoke(target, arguments);
}
private static T GetPrivateField<T>(object target, string fieldName)
where T : class
{
var field = target.GetType().GetField(
fieldName,
BindingFlags.Instance | BindingFlags.NonPublic);
Assert.That(field, Is.Not.Null, $"Expected private field '{fieldName}'.");
return field.GetValue(target) as T;
}
private static Mesh CreateMesh()
{
var mesh = new Mesh { name = "GazeCalibrationWindowMesh" };
mesh.vertices = new[]
{
new Vector3(-0.1f, 0f, 0f),
new Vector3(0.1f, 0f, 0f),
new Vector3(0f, 0.1f, 0f)
};
mesh.triangles = new[] { 0, 1, 2 };
mesh.AddBlendShapeFrame(
"eyeLookUpLeft",
100f,
new[] { Vector3.up * 0.01f, Vector3.up * 0.01f, Vector3.up * 0.01f },
new Vector3[3],
new Vector3[3]);
return mesh;
}
}
}

View File

@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: de38c6c9919441369ab6564cd13399f7

View File

@ -0,0 +1,149 @@
using System;
using System.Collections.Generic;
using NUnit.Framework;
using UnityEngine;
namespace Streamingle.Gaze.Tests
{
public sealed class GazeDirectionSolverTests
{
[Test]
public void Calculate_ClampsAndNormalizesAsymmetricRanges()
{
var result = GazeDirectionSolver.Calculate(
Quaternion.identity,
GazeDirectionSolver.DirectionFromAngles(60f, 30f),
-20f,
40f,
-10f,
20f);
Assert.That(result.RawYaw, Is.EqualTo(60f).Within(0.01f));
Assert.That(result.RawPitch, Is.EqualTo(30f).Within(0.01f));
Assert.That(result.ClampedYaw, Is.EqualTo(40f).Within(0.01f));
Assert.That(result.ClampedPitch, Is.EqualTo(20f).Within(0.01f));
Assert.That(result.NormalizedYaw, Is.EqualTo(1f).Within(0.001f));
Assert.That(result.NormalizedPitch, Is.EqualTo(1f).Within(0.001f));
Assert.That(result.WasClamped, Is.True);
}
[Test]
public void Calculate_UsesCharacterLocalReference()
{
var reference = Quaternion.Euler(5f, 73f, 0f);
var localDirection = GazeDirectionSolver.DirectionFromAngles(-10f, -5f);
var result = GazeDirectionSolver.Calculate(
reference,
reference * localDirection,
-20f,
40f,
-10f,
20f);
Assert.That(result.RawYaw, Is.EqualTo(-10f).Within(0.01f));
Assert.That(result.RawPitch, Is.EqualTo(-5f).Within(0.01f));
Assert.That(result.NormalizedYaw, Is.EqualTo(-0.5f).Within(0.001f));
Assert.That(result.NormalizedPitch, Is.EqualTo(-0.5f).Within(0.001f));
Assert.That(result.WasClamped, Is.False);
}
[Test]
public void Profile_BilinearlyInterpolatesTheThreeByThreeGrid()
{
var profile = ScriptableObject.CreateInstance<BlendshapeGazeProfile>();
try
{
profile.ReplaceChannels(new[]
{
new GazeBlendShapeChannel("eyeLookUpLeft")
});
for (var index = 0; index < 9; index++)
profile.SetSample((GazeCalibrationPoint)index, new[] { (float)index });
var output = new float[1];
var direction = DirectionWithNormalizedCoordinates(-0.5f, -0.5f);
Assert.That(profile.TryEvaluate(direction, false, output), Is.True);
// Bottom-left/center cell: lerp of sample values 0,1,3,4.
Assert.That(output[0], Is.EqualTo(2f).Within(0.001f));
direction = DirectionWithNormalizedCoordinates(0.5f, 0.5f);
Assert.That(profile.TryEvaluate(direction, false, output), Is.True);
// Center/top-right cell: lerp of sample values 4,5,7,8.
Assert.That(output[0], Is.EqualTo(6f).Within(0.001f));
}
finally
{
UnityEngine.Object.DestroyImmediate(profile);
}
}
[Test]
public void Profile_ExcludesCorrectiveChannelsUnlessRequested()
{
var profile = ScriptableObject.CreateInstance<BlendshapeGazeProfile>();
try
{
profile.ReplaceChannels(new List<GazeBlendShapeChannel>
{
new GazeBlendShapeChannel("eyeLookUpLeft"),
new GazeBlendShapeChannel("eyeBlinkLeft", GazeChannelUsage.Corrective)
});
for (var index = 0; index < 9; index++)
profile.SetSample((GazeCalibrationPoint)index, new[] { 10f, 20f });
var output = new float[2];
Assert.That(profile.TryEvaluate(DirectionWithNormalizedCoordinates(0f, 0f), false, output), Is.True);
Assert.That(output[0], Is.EqualTo(10f));
Assert.That(float.IsNaN(output[1]), Is.True);
Assert.That(profile.TryEvaluate(DirectionWithNormalizedCoordinates(0f, 0f), true, output), Is.True);
Assert.That(output[1], Is.EqualTo(20f));
}
finally
{
UnityEngine.Object.DestroyImmediate(profile);
}
}
[Test]
public void Profile_DoesNotEvaluateAnIncompleteCalibration()
{
var profile = ScriptableObject.CreateInstance<BlendshapeGazeProfile>();
try
{
profile.ReplaceChannels(new[]
{
new GazeBlendShapeChannel("eyeLookUpLeft")
});
profile.SetSample(GazeCalibrationPoint.Center, new[] { 50f });
Assert.That(
profile.TryEvaluate(
DirectionWithNormalizedCoordinates(0f, 0f),
false,
new float[1]),
Is.False);
}
finally
{
UnityEngine.Object.DestroyImmediate(profile);
}
}
private static GazeDirectionResult DirectionWithNormalizedCoordinates(float yaw, float pitch)
{
return new GazeDirectionResult(
Vector3.forward,
Vector3.forward,
0f,
0f,
0f,
0f,
yaw,
pitch);
}
}
}

Some files were not shown because too many files have changed in this diff Show More