\n\n### Features\n\n- Virtual Background\n- Share screen as virtual background\n- Share movie as virtual background\n- (not yet)\n\n## Demo\nhttps://flect-lab-web.s3-us-west-2.amazonaws.com/001_virtual_background/index.html\n\n### Demo source code\nhttps://github.com/dannadori/virtual-background-demo\n\n## Requirement\nThis library is tested only with react.\n\n## Usage\n### Sample Code\n\n```\nimport * as React from 'react';\nimport { LocalVideoEffectors } from 'local-video-effector'\n/**\n * Main Component\n */\nclass App extends React.Component {\n localCanvasRef = React.createRef()\n localVideoEffectors = new LocalVideoEffectors(null)\n componentDidMount() {\n this.localVideoEffectors.cameraEnabled = true\n this.localVideoEffectors.virtualBackgroundEnabled = true\n this.localVideoEffectors.virtualBackgroundImagePath = \"/resources/vbg/pic1.jpg\"\n this.localVideoEffectors.selectInputVideoDevice(\"\").then(() => {\n requestAnimationFrame(() => this.drawVideoCanvas())\n })\n }\n\n drawVideoCanvas = () => {\n this.localVideoEffectors.doEffect(960,640)\n if (this.localCanvasRef.current !== null) {\n if (this.localVideoEffectors.outputWidth !== 0 && this.localVideoEffectors.outputHeight !== 0) {\n this.localCanvasRef.current.height = (this.localCanvasRef.current.width / this.localVideoEffectors.outputWidth) * this.localVideoEffectors.outputHeight\n const ctx = this.localCanvasRef.current.getContext(\"2d\")!\n ctx.drawImage(this.localVideoEffectors.outputCanvas, 0, 0, this.localCanvasRef.current.width, this.localCanvasRef.current.height)\n }\n }\n requestAnimationFrame(() => this.drawVideoCanvas())\n }\n\n\n render() {\n return (\n
\n \n
\n )\n }\n}\n\n\nexport default App;\n```\n### constructor\nYou should instansiate LocalVideoEffectors.\n\n```\nlocalVideoEffectors = new LocalVideoEffectors(null)\n```\n\n### componentDidMount\nYou shuold configure the instance.\n```\n this.localVideoEffectors.cameraEnabled = true\n this.localVideoEffectors.virtualBackgroundEnabled = true\n this.localVideoEffectors.virtualBackgroundImagePath = \"/resources/vbg/pic1.jpg\"\n```\n\n\nSelect camera device. If you input empty string as parameter, default camera is selected.\nthen draw image.\n```\n this.localVideoEffectors.selectInputVideoDevice(\"\").then(() => {\n requestAnimationFrame(() => this.drawVideoCanvas())\n })\n```\n\nDrawing. doEffect accepts the resolution in which this libary apply the effects. So, output image size is the same as this parametar. And the performance depends on this parameter. Output frame size is the same size as background image. If you chnage the size, you can pass the 3rd and 4th parameters as width and height.\n\n```\n drawVideoCanvas = () => {\n this.localVideoEffectors.doEffect(960,640)\n if (this.localCanvasRef.current !== null) {\n if (this.localVideoEffectors.outputWidth !== 0 && this.localVideoEffectors.outputHeight !== 0) {\n this.localCanvasRef.current.height = (this.localCanvasRef.current.width / this.localVideoEffectors.outputWidth) * this.localVideoEffectors.outputHeight\n const ctx = this.localCanvasRef.current.getContext(\"2d\")!\n ctx.drawImage(this.localVideoEffectors.outputCanvas, 0, 0, this.localCanvasRef.current.width, this.localCanvasRef.current.height)\n }\n }\n requestAnimationFrame(() => this.drawVideoCanvas())\n }\n```\n\n### Advanced\n#### Blur\nYou can change blur with maskBlurAmount\n\n```\nthis.localVideoEffectors.maskBlurAmount = blur\n```\n\nhttps://virtual-background-bodypix.herokuapp.com/index.html?blur=2\n\n#### Model\nYou can chnage model with constructor.\nAcceptable parameter is \"null\", \"ModelConfigMobileNetV1\", \"ModelConfigResNet\"\n```\nthis.localVideoEffectors = new LocalVideoEffectors(ModelConfigMobileNetV1)\n```\n- ModelConfigResNet\nhttps://virtual-background-bodypix.herokuapp.com/index.html?blur=0&model=ResNet\n\n- ModelConfigMobileNetV1\nhttps://virtual-background-bodypix.herokuapp.com/index.html?blur=0&model=MobileNetV1\n\n#### Sharing screen or movie as virtual background\nThis package have the function to use the mediastream as virtual background. You can set the Mediastrem as below example. \n\nThis example shows the how to change the virtual background. These function is called by your UI button such as onClick or so.\n\n```\n\n setBGImage = () => {\n this.localVideoEffectors!.virtualBackgroundImageElement = this.virtualBGImage\n this.setState({foregroundSizeChange: false})\n this.shareVideoElementRef.current!.pause()\n }\n\n // For SharedDisplay\n sharedDisplaySelected = () => {\n const streamConstraints = {\n // frameRate: {\n // max: 15,\n // },\n }\n // @ts-ignore https://github.com/microsoft/TypeScript/issues/31821\n navigator.mediaDevices.getDisplayMedia().then(media => {\n this.localVideoEffectors!.virtualBackgroundStream = media\n this.setState({foregroundSizeChange: true})\n this.shareVideoElementRef.current!.pause()\n })\n }\n\n // For SharedVideo\n sharedVideoSelected = (e: any) => {\n const path = URL.createObjectURL(e.target.files[0]);\n console.log(path)\n this.shareVideoElementRef.current!.src = path\n this.shareVideoElementRef.current!.play()\n\n setTimeout(\n async () => {\n // @ts-ignore\n const mediaStream: MediaStream = await this.shareVideoElementRef.current!.captureStream()\n this.localVideoEffectors!.virtualBackgroundStream = mediaStream\n this.setState({foregroundSizeChange: true})\n }\n , 3000); // I don't know but we need some seconds to restart video share....\n }\n\n```\n\n#### Virtual Foreground\nYou can use mask to hide user face by line drawing or ascii art.\n\n
\n\n
\n\n\n```\nthis.localVideoEffectors!.foregroundType = ForegroundType.Canny\n```\n\nor\n\n```\nthis.localVideoEffectors!.foregroundType = ForegroundType.Ascii\n```\n\n### iPhone safari\nIf you use the safari on iPhone, your video element must be run by clicking start button. This means video element must be controllable from your code directory. This package provide the method to set the HTMLVideoElement you defined.\n\nUsage example:\nIn this example, render method of ReactComponent return HTMLVideoElement and Button which sets the HTMLElement to the this package(LocalVideoEeffectors).\n\n```\n return (\n
![](\"./doc/demo_basic.gif\")
\n![](\"./doc/demo_vfg.gif\")
\n