Konwersja YUV NA RGB przez fragment shader

Mam problem z konwersją podglądu kamery w Androidzie z formatu YUV NA RGB. Celem konwersji jest zastosowanie niektórych efektów. Staram się przekonwertować przez fragment shader ponieważ konwersja przez natywny kod jest powolna (około 14fps). Odniesienie, którego użyłem to http://jyrom.tistory.com/m/post/view/id/187 . staram się przenieść ten kod na platformę Android, ale wynik jest czarno-zielone prostokąty. Ale mogę oglądać jakąś formę przez wyjście, które otrzymuję. Czy mógłbyś spróbować pomóż mi rozwiązać ten problem. Uważam, że jest to popularny problem: zastosuj efekty do podglądu kamery. Podaję również link do mojego projektu do testów: https://dl.dropbox.com/u/12829395/application/FilterGL/FilterGL.zip . Dziękuję.
Aktualizacja:
To jest moja metoda onPreviewFrame :

public void onPreviewFrame(byte[] data, Camera camera) {
    yBuffer.put(data);
    yBuffer.position(0);

    System.arraycopy(data, U_INDEX, uData, 0, LENGTH_4 * 2);
    uBuffer.put(uData);
    uBuffer.position(0);

    System.arraycopy(data, V_INDEX, vData, 0, LENGTH_4);
    vBuffer.put(vData);
    vBuffer.position(0);
}

Tak przyporządkowuję tablice bajtów do tekstury OpenGL w metodzie onDrawFrame :

    GLES20.glUniform1i(yTexture, 1);
    GLES20.glTexImage2D(   GLES20.GL_TEXTURE_2D, 0, GLES20.GL_LUMINANCE,
            320, 240, 0, GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE, yBuffer);
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);

    GLES20.glUniform1i(uTexture, 2);
    GLES20.glTexImage2D(   GLES20.GL_TEXTURE_2D, 0, GLES20.GL_LUMINANCE,
            160, 120, 0, GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE, uBuffer);
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);

    GLES20.glUniform1i(vTexture, 3);
    GLES20.glTexImage2D(   GLES20.GL_TEXTURE_2D, 0, GLES20.GL_LUMINANCE,
            160, 120, 0, GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE, vBuffer);
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);

A to jest mój fragment shader code:

#ifdef GL_ES
precision highp float;
#endif

varying vec2 v_texCoord;
uniform sampler2D y_texture;
uniform sampler2D u_texture;
uniform sampler2D v_texture;

void main()
{   
    float nx,ny,r,g,b,y,u,v;
    nx=v_texCoord.x;
    ny=v_texCoord.y;
    y=texture2D(y_texture,v_texCoord).r;
    u=texture2D(u_texture,v_texCoord).r;
    v=texture2D(v_texture,v_texCoord).r;

    y=1.1643*(y-0.0625);
    u=u-0.5;
    v=v-0.5;

    r=y+1.5958*v;
    g=y-0.39173*u-0.81290*v;
    b=y+2.017*u;

    gl_FragColor = vec4(r,g,b,1.0);
}
Author: Paul Gregoire, 2012-08-26

5 answers

Nie jestem pewien, czy już to naprawiłeś problem.My ODPOWIEDŹ

  1. domyślnie wyjściem kamery jest NV12, ale w fragment shader YUV do RGB używasz YV12 - > RGB. Będziesz musiał zrobić setPreviewFormat(ImageFormat.YV12);, lub może być użyty jakiś inny shader
  2. Są 3 tekstury, upewnij się, że robisz

    GLES20.glActiveTexture(GLES20.GL_TEXTURE2); GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, muTextureName)

    Przed wywołaniem dowolnego glTexImage2D. i glTexSubImage2D

  3. Możesz również używać glTexSubImage2D z każdą klatką i glTexImage2D raz.

  4. Rozmiar U I V jest taki sam, przynajmniej dla YV12,

    System.arraycopy(data, U_INDEX, uData, 0, LENGTH_4 * 2);

    Powinno być System.arraycopy(data, U_INDEX, uData, 0, LENGTH_4); zmień odpowiednio rozmiar w kodzie.

 3
Author: psharma,
Warning: date(): Invalid date.timezone value 'Europe/Kyiv', we selected the timezone 'UTC' for now. in /var/www/agent_stack/data/www/doraprojects.net/template/agent.layouts/content.php on line 54
2012-12-18 16:41:24

Nie wiem, czy rozwiązałeś swój problem.

Użyłem Twojego kodu i rozwiązałem w tym trybie.

public class MyRenderer implements Renderer{
public static final int recWidth = Costanti.recWidth;
public static final int recHeight = Costanti.recHeight;

private static final int U_INDEX = recWidth*recHeight;
private static final int V_INDEX = recWidth*recHeight*5/4;
private static final int LENGTH = recWidth*recHeight;
private static final int LENGTH_4 = recWidth*recHeight/4;

private int previewFrameWidth = 256;
private int previewFrameHeight = 256;

private int[] yTextureNames;
private int[] uTextureNames;
private int[] vTextureNames;

private MainActivity activity;

private FloatBuffer mVertices;
private ShortBuffer mIndices;

private int mProgramObject;
private int mPositionLoc;
private int mTexCoordLoc;

private int yTexture;
private int uTexture;
private int vTexture;

private final float[] mVerticesData = { -1.f, 1.f, 0.0f, // Position 0
        0.0f, 0.0f, // TexCoord 0
        -1.f, -1.f, 0.0f, // Position 1
        0.0f, 1.0f, // TexCoord 1
        1.f, -1.f, 0.0f, // Position 2
        1.0f, 1.0f, // TexCoord 2
        1.f, 1.f, 0.0f, // Position 3
        1.0f, 0.0f // TexCoord 3
};
private final short[] mIndicesData = { 0, 1, 2, 0, 2, 3 };

private ByteBuffer yBuffer;
private ByteBuffer uBuffer;
private ByteBuffer vBuffer;

private IntBuffer frameBuffer;
private IntBuffer renderBuffer;
private IntBuffer parameterBufferWidth;
private IntBuffer parameterBufferHeigth;

byte[] ydata = new byte[LENGTH];
byte[] uData = new byte[LENGTH_4];
byte[] vData = new byte[LENGTH_4];

public MyRenderer(MainActivity activity) {
    this.activity = activity;

    mVertices = ByteBuffer.allocateDirect(mVerticesData.length * 4)
            .order(ByteOrder.nativeOrder()).asFloatBuffer();
    mVertices.put(mVerticesData).position(0);

    mIndices = ByteBuffer.allocateDirect(mIndicesData.length * 2)
            .order(ByteOrder.nativeOrder()).asShortBuffer();
    mIndices.put(mIndicesData).position(0);

    yBuffer = MyGraphUtils.makeByteBuffer(LENGTH);
    uBuffer = MyGraphUtils.makeByteBuffer(LENGTH_4/* * 2*/);
    vBuffer = MyGraphUtils.makeByteBuffer(LENGTH_4);
}

@Override
public void onSurfaceChanged(GL10 gl, int width, int height) {
    GLES20.glActiveTexture(GLES20.GL_ACTIVE_TEXTURE);
    GLES20.glViewport(0, 0, width, height);
}

@Override
public void onSurfaceCreated(GL10 gl, EGLConfig config) {
    Log.d("debug", "on surface created");
    // Define a simple shader program for our point.
    final String vShaderStr = readTextFileFromRawResource(activity, R.raw.v_simple);
    final String fShaderStr = readTextFileFromRawResource(activity, R.raw.f_convert);
    frameBuffer = IntBuffer.allocate(1);
    renderBuffer= IntBuffer.allocate(1);

    GLES20.glEnable(GLES20.GL_TEXTURE_2D);

    GLES20.glGenFramebuffers(1, frameBuffer);
    GLES20.glGenRenderbuffers(1, renderBuffer);
    GLES20.glActiveTexture(GLES20.GL_ACTIVE_TEXTURE);
    GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, frameBuffer.get(0));
    GLES20.glClear(0);
    GLES20.glBindRenderbuffer(GLES20.GL_RENDERBUFFER, renderBuffer.get(0));     

    GLES20.glRenderbufferStorage(GLES20.GL_RENDERBUFFER, GLES20.GL_DEPTH_COMPONENT16,
                                 320, 240);

    parameterBufferHeigth = IntBuffer.allocate(1);
    parameterBufferWidth = IntBuffer.allocate(1);
    GLES20.glGetRenderbufferParameteriv(GLES20.GL_RENDERBUFFER, GLES20.GL_RENDERBUFFER_WIDTH, parameterBufferWidth);
    GLES20.glGetRenderbufferParameteriv(GLES20.GL_RENDERBUFFER, GLES20.GL_RENDERBUFFER_HEIGHT, parameterBufferHeigth);
    GLES20.glFramebufferRenderbuffer(GLES20.GL_FRAMEBUFFER, GLES20.GL_COLOR_ATTACHMENT0, GLES20.GL_RENDERBUFFER, renderBuffer.get(0));
    if (GLES20.glCheckFramebufferStatus(GLES20.GL_FRAMEBUFFER)!=GLES20.GL_FRAMEBUFFER_COMPLETE){
        Log.d("debug", "gl frame buffer status != frame buffer complete");
    }
    GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, 0);
    GLES20.glClear(0);

    mProgramObject = loadProgram(vShaderStr, fShaderStr);

    // Get the attribute locations
    mPositionLoc = GLES20.glGetAttribLocation(mProgramObject, "a_position");
    mTexCoordLoc = GLES20.glGetAttribLocation(mProgramObject, "a_texCoord");

    GLES20.glEnable(GLES20.GL_TEXTURE_2D);
    yTexture = GLES20.glGetUniformLocation(mProgramObject, "y_texture");
    yTextureNames = new int[1];
    GLES20.glGenTextures(1, yTextureNames, 0);
    int yTextureName = yTextureNames[0];

    GLES20.glEnable(GLES20.GL_TEXTURE_2D);
    uTexture = GLES20.glGetUniformLocation(mProgramObject, "u_texture");
    uTextureNames = new int[1];
    GLES20.glGenTextures(1, uTextureNames, 0);
    int uTextureName = uTextureNames[0];

    GLES20.glEnable(GLES20.GL_TEXTURE_2D);
    vTexture = GLES20.glGetUniformLocation(mProgramObject, "v_texture");
    vTextureNames = new int[1];
    GLES20.glGenTextures(1, vTextureNames, 0);
    int vTextureName = vTextureNames[0];

    GLES20.glClearColor(1.0f, 0.0f, 0.0f, 0.0f);
}

@Override
public final void onDrawFrame(GL10 gl) {
    Log.d("debug", "on Draw frame");
    // Clear the color buffer
    GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT);

    // Use the program object
    GLES20.glUseProgram(mProgramObject);

    // Load the vertex position
    mVertices.position(0);
    GLES20.glVertexAttribPointer(mPositionLoc, 3, GLES20.GL_FLOAT, false, 5*4, mVertices);
    // Load the texture coordinate
    mVertices.position(3);
    GLES20.glVertexAttribPointer(mTexCoordLoc, 2, GLES20.GL_FLOAT, false, 5*4, mVertices);

    GLES20.glEnableVertexAttribArray(mPositionLoc);
    GLES20.glEnableVertexAttribArray(mTexCoordLoc);

    GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, yTextureNames[0]);
    GLES20.glTexImage2D(   GLES20.GL_TEXTURE_2D, 0, GLES20.GL_LUMINANCE,
            320, 240, 0, GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE, yBuffer);
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
    GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
    GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);
    GLES20.glActiveTexture(GLES20.GL_TEXTURE1);
    GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, yTextureNames[0]);
    GLES20.glUniform1i(yTexture, 0);

    GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, uTextureNames[0]);
    GLES20.glTexImage2D(   GLES20.GL_TEXTURE_2D, 0, GLES20.GL_LUMINANCE,
            160, 120, 0, GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE, uBuffer);
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
    GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
    GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);
    GLES20.glActiveTexture(GLES20.GL_TEXTURE1+2);
    GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, uTextureNames[0]);
    GLES20.glUniform1i(uTexture, 2);

    GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, vTextureNames[0]);
    GLES20.glTexImage2D(   GLES20.GL_TEXTURE_2D, 0, GLES20.GL_LUMINANCE,
            160, 120, 0, GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE, vBuffer);
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
    GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
    GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);
    GLES20.glActiveTexture(GLES20.GL_TEXTURE1+1);
    GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, vTextureNames[0]);
    GLES20.glUniform1i(vTexture, 1);

    GLES20.glDrawElements(GLES20.GL_TRIANGLES, 6, GLES20.GL_UNSIGNED_SHORT, mIndices);
}



public void setPreviewFrameSize(int realWidth, int realHeight) {
    previewFrameHeight = realHeight;
    previewFrameWidth = realWidth;
}

public static String readTextFileFromRawResource(final Context context, final int resourceId) {
    final InputStream inputStream = context.getResources().openRawResource(resourceId);
    final InputStreamReader inputStreamReader = new InputStreamReader(inputStream);
    final BufferedReader bufferedReader = new BufferedReader(inputStreamReader);

    String nextLine;
    final StringBuilder body = new StringBuilder();

    try {
        while ((nextLine = bufferedReader.readLine()) != null) {
            body.append(nextLine);
            body.append('\n');
        }
    } catch (IOException e) {
        return null;
    }

    return body.toString();
}

public static int loadShader(int type, String shaderSrc) {
    int shader;
    int[] compiled = new int[1];

    // Create the shader object
    shader = GLES20.glCreateShader(type);
    if (shader == 0) {
        return 0;
    }
    // Load the shader source
    GLES20.glShaderSource(shader, shaderSrc);
    // Compile the shader
    GLES20.glCompileShader(shader);
    // Check the compile status
    GLES20.glGetShaderiv(shader, GLES20.GL_COMPILE_STATUS, compiled, 0);

    if (compiled[0] == 0) {
        Log.e("ESShader", GLES20.glGetShaderInfoLog(shader));
        GLES20.glDeleteShader(shader);
        return 0;
    }
    return shader;
}

public static int loadProgram(String vertShaderSrc, String fragShaderSrc) {
    int vertexShader;
    int fragmentShader;
    int programObject;
    int[] linked = new int[1];

    // Load the vertex/fragment shaders
    vertexShader = loadShader(GLES20.GL_VERTEX_SHADER, vertShaderSrc);
    if (vertexShader == 0) {
        return 0;
    }

    fragmentShader = loadShader(GLES20.GL_FRAGMENT_SHADER, fragShaderSrc);
    if (fragmentShader == 0) {
        GLES20.glDeleteShader(vertexShader);
        return 0;
    }

    // Create the program object
    programObject = GLES20.glCreateProgram();

    if (programObject == 0) {
        return 0;
    }

    GLES20.glAttachShader(programObject, vertexShader);
    GLES20.glAttachShader(programObject, fragmentShader);

    // Link the program
    GLES20.glLinkProgram(programObject);

    // Check the link status
    GLES20.glGetProgramiv(programObject, GLES20.GL_LINK_STATUS, linked, 0);

    if (linked[0] == 0) {
        Log.e("ESShader", "Error linking program:");
        Log.e("ESShader", GLES20.glGetProgramInfoLog(programObject));
        GLES20.glDeleteProgram(programObject);
        return 0;
    }

    // Free up no longer needed shader resources
    GLES20.glDeleteShader(vertexShader);
    GLES20.glDeleteShader(fragmentShader);

    return programObject;
}

@Override
public void onPreviewFrame(byte[] data, Camera camera) {

    System.arraycopy(data, 0, ydata, 0, LENGTH);
    yBuffer.put(ydata);
    yBuffer.position(0);

    System.arraycopy(data, U_INDEX, uData, 0, LENGTH_4);
    uBuffer.put(uData);
    uBuffer.position(0);

    System.arraycopy(data, V_INDEX, vData, 0, LENGTH_4);
    vBuffer.put(vData);
    vBuffer.position(0);
}   

}

 3
Author: ZanoOnStack,
Warning: date(): Invalid date.timezone value 'Europe/Kyiv', we selected the timezone 'UTC' for now. in /var/www/agent_stack/data/www/doraprojects.net/template/agent.layouts/content.php on line 54
2013-07-12 13:10:52

Aby uzyskać najszybszy i najbardziej zoptymalizowany sposób, wystarczy użyć wspólnego rozszerzenia GL

//Fragment Shader
#extension GL_OES_EGL_image_external : require
uniform samplerExternalOES u_Texture;

Niż w Javie

surfaceTexture = new SurfaceTexture(textureIDs[0]);
try {
   someCamera.setPreviewTexture(surfaceTexture);
} catch (IOException t) {
   Log.e(TAG, "Cannot set preview texture target!");
}

someCamera.startPreview();

private static final int GL_TEXTURE_EXTERNAL_OES = 0x8D65;

In Java GL Thread

GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
GLES20.glBindTexture(GL_TEXTURE_EXTERNAL_OES, textureIDs[0]);
GLES20.glUniform1i(uTextureHandle, 0);

Konwersja kolorów jest już wykonywana za Ciebie. Możesz robić, co chcesz, dokładnie w cieniowaniu fragmentów.

Mam nadzieję, że zaoszczędzisz trochę czasu na badaniach.
 2
Author: fky,
Warning: date(): Invalid date.timezone value 'Europe/Kyiv', we selected the timezone 'UTC' for now. in /var/www/agent_stack/data/www/doraprojects.net/template/agent.layouts/content.php on line 54
2014-03-19 14:43:18

W końcu twój projekt wyświetla podgląd kamery. Znalazłem 2 problemy: 1. Przed związaniem i zmianą charakterystyki powierzchni należy zadzwonić do GLES20.glActiveTexture(GLES20.surfacenumber); 2. Ważniejszym i ukrytym problemem jest to, że GLES20glTexImage2D() nie działa z szerokością i wysokością, które nie są potęgą 2 liczb. Po wczytaniu tekstury o rozmiarze np. 1024X1024 należy wywołać GLES20.glTexSubImage2D ()

Powodzenia!

 1
Author: bahakz,
Warning: date(): Invalid date.timezone value 'Europe/Kyiv', we selected the timezone 'UTC' for now. in /var/www/agent_stack/data/www/doraprojects.net/template/agent.layouts/content.php on line 54
2013-03-04 19:05:28

Zastosowałem formularz rozwiązania Jak renderować obraz z kamery YUV-Nv21 Androida w tle w libgdx z OpenGLES 2.0 w czasie rzeczywistym? do projektu udostępnionego w pytaniu i dostał działający projekt. Jeśli podobnie jak ja poszukujesz kodu samouczka, który wykonuje konwersję YUV NA RGB przez fragment shader, możesz po prostu wykonać następujące kroki, aby uzyskać działający przykład.

  1. Pobierz projekt https://dl.dropbox.com/u/12829395/application/FilterGL/FilterGL.zip oraz rozpakuj.
  2. Zastąp plik .java i res / raw / f_convert.glsl w całości wg kodu udostępnionego poniżej.
  3. Otwórz projekt w Eclipse, lub zaimportuj projekt do Android Studio .

Główne zagadnienia kodeksu w pytaniu to:

  1. bez GLES20. glActiveTexture (GLES20.GL_TEXTURE1); , yBuffer nie jest przekazywany do GL.
  2. DANE YUV przyjmują format YUV-NV21, a u_texture i v_texture nie zostały przekazane i prawidłowo obsługiwane w cieniu. Zobacz ten post Aby uzyskać więcej informacji.

Teraz poprawiony kod: proszę zastąpić Glrenderera.java z

package com.filtergl.shader;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.FloatBuffer;
import java.nio.ShortBuffer;

import javax.microedition.khronos.egl.EGLConfig;
import javax.microedition.khronos.opengles.GL10;

import android.content.Context;
import android.hardware.Camera;
import android.hardware.Camera.PreviewCallback;
import android.opengl.GLES20;
import android.opengl.GLSurfaceView.Renderer;
import android.util.Log;

public class GLRenderer
implements Renderer, PreviewCallback {
    private static final int LENGTH = 76800;
    private static final int LENGTH_2 = 38400;

    private ActivityFilterGL activity;

    private FloatBuffer mVertices;
    private ShortBuffer mIndices;

    private int previewFrameWidth = 256;
    private int previewFrameHeight = 256;
    private int mProgramObject;
    private int mPositionLoc;
    private int mTexCoordLoc;
//  private int mSamplerLoc;
    private int yTexture;
    private int uTexture;
    private int vTexture;

    private final float[] mVerticesData = { -1.f, 1.f, 0.0f, // Position 0
            0.0f, 0.0f, // TexCoord 0
            -1.f, -1.f, 0.0f, // Position 1
            0.0f, 1.0f, // TexCoord 1
            1.f, -1.f, 0.0f, // Position 2
            1.0f, 1.0f, // TexCoord 2
            1.f, 1.f, 0.0f, // Position 3
            1.0f, 0.0f // TexCoord 3
    };

    private final short[] mIndicesData = { 0, 1, 2, 0, 2, 3 };

    private ByteBuffer frameData = null;
    private ByteBuffer yBuffer;
    private ByteBuffer uBuffer;

    public GLRenderer(ActivityFilterGL activity) {
        this.activity = activity;

        mVertices = ByteBuffer.allocateDirect(mVerticesData.length * 4)
                .order(ByteOrder.nativeOrder()).asFloatBuffer();
        mVertices.put(mVerticesData).position(0);

        mIndices = ByteBuffer.allocateDirect(mIndicesData.length * 2)
                .order(ByteOrder.nativeOrder()).asShortBuffer();
        mIndices.put(mIndicesData).position(0);

        yBuffer = GraphicsUtil.makeByteBuffer(LENGTH);
        uBuffer = GraphicsUtil.makeByteBuffer(LENGTH_2);
    }

    @Override
    public final void onDrawFrame(GL10 gl) {
        // Clear the color buffer
        GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT);

        // Use the program object
        GLES20.glUseProgram(mProgramObject);

        // Load the vertex position
        mVertices.position(0);
        GLES20.glVertexAttribPointer(mPositionLoc, 3, GLES20.GL_FLOAT, false, 5 * 4, mVertices);
        // Load the texture coordinate
        mVertices.position(3);
        GLES20.glVertexAttribPointer(mTexCoordLoc, 2, GLES20.GL_FLOAT, false, 5 * 4, mVertices);

        GLES20.glEnableVertexAttribArray(mPositionLoc);
        GLES20.glEnableVertexAttribArray(mTexCoordLoc);

        GLES20.glActiveTexture(GLES20.GL_TEXTURE1);

        GLES20.glUniform1i(yTexture, 1);
        GLES20.glTexImage2D(   GLES20.GL_TEXTURE_2D, 0, GLES20.GL_LUMINANCE,
                320, 240, 0, GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE, yBuffer);
        GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
        GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
        GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
        GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);

        GLES20.glActiveTexture(GLES20.GL_TEXTURE2);

        GLES20.glUniform1i(uTexture, 2);
        GLES20.glTexImage2D(   GLES20.GL_TEXTURE_2D, 0, GLES20.GL_LUMINANCE_ALPHA,
                160, 120, 0, GLES20.GL_LUMINANCE_ALPHA, GLES20.GL_UNSIGNED_BYTE, uBuffer);
        GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
        GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
        GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
        GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);

        GLES20.glDrawElements(GLES20.GL_TRIANGLES, 6, GLES20.GL_UNSIGNED_SHORT, mIndices);
    }

    @Override
    public void onSurfaceChanged(GL10 gl, int width, int height) {
        GLES20.glViewport(0, 0, width, height);
    }

    @Override
    public void onSurfaceCreated(GL10 gl, EGLConfig config) {

        // Define a simple shader program for our point.
        final String vShaderStr = readTextFileFromRawResource(activity, R.raw.v_simple);
        final String fShaderStr = readTextFileFromRawResource(activity, R.raw.f_convert);

        // Load the shaders and get a linked program object
        mProgramObject = loadProgram(vShaderStr, fShaderStr);

        // Get the attribute locations
        mPositionLoc = GLES20.glGetAttribLocation(mProgramObject, "a_position");
        mTexCoordLoc = GLES20.glGetAttribLocation(mProgramObject, "a_texCoord");

        GLES20.glEnable(GLES20.GL_TEXTURE_2D);
        yTexture = GLES20.glGetUniformLocation(mProgramObject, "y_texture");
        int[] yTextureNames = new int[1];
        GLES20.glGenTextures(1, yTextureNames, 0);
        int yTextureName = yTextureNames[0];
        GLES20.glActiveTexture(GLES20.GL_TEXTURE1);
        GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, yTextureName);

        GLES20.glEnable(GLES20.GL_TEXTURE_2D);
        uTexture = GLES20.glGetUniformLocation(mProgramObject, "u_texture");
        int[] uTextureNames = new int[1];
        GLES20.glGenTextures(1, uTextureNames, 0);
        int uTextureName = uTextureNames[0];
        GLES20.glActiveTexture(GLES20.GL_TEXTURE2);
        GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, uTextureName);

        // Set the background clear color to black.
        GLES20.glClearColor(1.0f, 0.0f, 0.0f, 0.0f);
    }

    public void setPreviewFrameSize(int realWidth, int realHeight) {
        previewFrameHeight = realHeight;
        previewFrameWidth = realWidth;

//      frameData = GraphicsUtil.makeByteBuffer(previewFrameHeight * previewFrameWidth * 3);
    }

    public static String readTextFileFromRawResource(final Context context, final int resourceId) {
        final InputStream inputStream = context.getResources().openRawResource(resourceId);
        final InputStreamReader inputStreamReader = new InputStreamReader(inputStream);
        final BufferedReader bufferedReader = new BufferedReader(inputStreamReader);

        String nextLine;
        final StringBuilder body = new StringBuilder();

        try {
            while ((nextLine = bufferedReader.readLine()) != null) {
                body.append(nextLine);
                body.append('\n');
            }
        } catch (IOException e) {
            return null;
        }

        return body.toString();
    }

    public static int loadShader(int type, String shaderSrc) {
        int shader;
        int[] compiled = new int[1];

        // Create the shader object
        shader = GLES20.glCreateShader(type);
        if (shader == 0) {
            return 0;
        }
        // Load the shader source
        GLES20.glShaderSource(shader, shaderSrc);
        // Compile the shader
        GLES20.glCompileShader(shader);
        // Check the compile status
        GLES20.glGetShaderiv(shader, GLES20.GL_COMPILE_STATUS, compiled, 0);

        if (compiled[0] == 0) {
            Log.e("ESShader", GLES20.glGetShaderInfoLog(shader));
            GLES20.glDeleteShader(shader);
            return 0;
        }
        return shader;
    }

    public static int loadProgram(String vertShaderSrc, String fragShaderSrc) {
        int vertexShader;
        int fragmentShader;
        int programObject;
        int[] linked = new int[1];

        // Load the vertex/fragment shaders
        vertexShader = loadShader(GLES20.GL_VERTEX_SHADER, vertShaderSrc);
        if (vertexShader == 0) {
            return 0;
        }

        fragmentShader = loadShader(GLES20.GL_FRAGMENT_SHADER, fragShaderSrc);
        if (fragmentShader == 0) {
            GLES20.glDeleteShader(vertexShader);
            return 0;
        }

        // Create the program object
        programObject = GLES20.glCreateProgram();

        if (programObject == 0) {
            return 0;
        }

        GLES20.glAttachShader(programObject, vertexShader);
        GLES20.glAttachShader(programObject, fragmentShader);

        // Link the program
        GLES20.glLinkProgram(programObject);

        // Check the link status
        GLES20.glGetProgramiv(programObject, GLES20.GL_LINK_STATUS, linked, 0);

        if (linked[0] == 0) {
            Log.e("ESShader", "Error linking program:");
            Log.e("ESShader", GLES20.glGetProgramInfoLog(programObject));
            GLES20.glDeleteProgram(programObject);
            return 0;
        }

        // Free up no longer needed shader resources
        GLES20.glDeleteShader(vertexShader);
        GLES20.glDeleteShader(fragmentShader);

        return programObject;
    }

    @Override
    public void onPreviewFrame(byte[] data, Camera camera) {
        yBuffer.put(data, 0, LENGTH);
        yBuffer.position(0);

        uBuffer.put(data, LENGTH, LENGTH/2);
        uBuffer.position(0);
    }

}

I zastąp f_convert.glsl z

#ifdef GL_ES
precision highp float;
#endif

varying vec2 v_texCoord;
uniform sampler2D y_texture;
uniform sampler2D u_texture;

 void main()
{   
    float r, g, b, y, u, v;

    //We had put the Y values of each pixel to the R,G,B components by
    //GL_LUMINANCE, that's why we're pulling it from the R component,
    //we could also use G or B
    y = texture2D(y_texture, v_texCoord).r;

    //We had put the U and V values of each pixel to the A and R,G,B
    //components of the texture respectively using GL_LUMINANCE_ALPHA.
    //Since U,V bytes are interspread in the texture, this is probably
    //the fastest way to use them in the shader
    u = texture2D(u_texture, v_texCoord).a - 0.5;
    v = texture2D(u_texture, v_texCoord).r - 0.5;

    //The numbers are just YUV to RGB conversion constants
    r = y + 1.13983*v;
    g = y - 0.39465*u - 0.58060*v;
    b = y + 2.03211*u;

    gl_FragColor = vec4(r,g,b,1.0);
}
 0
Author: Peter,
Warning: date(): Invalid date.timezone value 'Europe/Kyiv', we selected the timezone 'UTC' for now. in /var/www/agent_stack/data/www/doraprojects.net/template/agent.layouts/content.php on line 54
2017-05-23 12:08:58